- Code efficiency enhancements and bug fixes for plugins.
- Significant UI/UX enhancements.
- Implementation of MFA.
- Due to the above change, if you are upgrading from version 3.6, an update has been made to the backend database. In the installation directory, there is a file called "3.7_Upgrade.py", please copy your
config.jsonfile to this directory and run the script to update it to the latest standard. Running it will update the
userstable to be in line with the new version.
Scrummage is an OSINT tool that centralises search functionality from a bounty of powerful, publicly-available, third-party, OSINT websites. This project draws inspiration mainly from two other, open-source projects, including:
- The Scumblr project, which while is now deprecated, inspired this concept.
- The OSINT Framework project, which is a visualization tool, depicts a range of sites that can be used to search for a variety of things.
While at first glance the web application may not look all that different when compared to Scumblr, the copious amounts of plugins this tool comes with is mainly what makes this project unique, where the provided Python/Flask web application is just a simple, lightweight, and scalable way of providing users with the ability to manage large pools of results. The other main benefit this projects brags is a much simpler installation process, which is kept up to date, compared to Scumblr which is now deprecated.
Any feedback is welcome.
FOR INSTRUCTIONS REFER TO THE WIKI
An Overview of the Web Application
Some of the Many Available Scrummage Plugins
- Blockchain Search
- Domain Fuzzer
- Twitter Scraper
- Have I Been Pwned Search
- Ahmia Darkweb Search
- IP Stack Search
- Threat Crowd Search
- Yandex and Naver Search
- Vkontakte Search
- Vulners Search
- Built With Search
- YouTube Search
- Many more... Refer to the wiki page here for the full list.
The dashboard is the home screen which the application directs a user to when they log in. It provides a high-level chart which shows the amount of each results based on their result type. It does this for each kind of finding. However, if a graph doesn’t load, this is most likely due to none of the results being in that category, I.e if there are no closed results, no graph will appear under “Overview of Closed Results”.
The events page shows anything that changes within the web application, from logins, to failed login attempts, to any actions performed against a task. This assists with understanding what has recently been happening in the web app, and can assist in matters such as detecting brute-force login attempts or tracking down who altered a task.
Note: This page only loads the latest 1000 events, for optimisation of the web application.
The results page simply shows results that have been created by a task. The results table shows the basic metadata of the result, but also provides a “Details” button which can be used to investigate the result further. As mentioned all results have some kind of output file, if a result is a link the file will be a copy of the HTML of the page. Furthermore, screenshot functionality is provided to assist in keeping a photographic record of a result. Both the output and screenshot file will be deleted if the result is deleted.
Note: This page only loads the latest 1000 results, for optimisation of the web application.
For optimisation purposes, the results table only displays some of the general information regarding a result, to investigate a result further, the user should use the Details button. The details page allows the user to view the soft copy of the result's link and provides the ability for a user to generate a screenshot.
The tasks page shows all created task and provides the ability for the user to run each task. This page doesn’t have a limit on tasks; however, don’t go crazy creating tasks, you can always add a list to a task, rather than having the same task created multiple times for one search. So really you shouldn’t have any more than 50 tasks. Tasks have to cache and log for each which can be found in the “protected/output” directory under the tasks name, ex. Google Search is called “google”. If you need to remove the cache, you can edit/delete the appropriate cache file.
All the plugins are open-source, free to individuals, just like the rest of the code. Furthermore, feel free to use the pre-existing libraries used in other plugins. If you are creating or editing a plugin, make sure to understand that when you run it for the first time, the web app may reload to reload the python cache. This is normal.
This page changes according to the user's privileges, if a user is an admin, they have the ability to change their password as well as other user's passwords, they can block and unblock users, demote and promote users' privileges, and of course, create new users and delete existing users.
Additionally, users with administrative privileges can check and edit input, output, and core configuration of the tool.
The account page looks as per below for administrative users:
The account page looks as per below for non-administrative users:
This concept was introduced in v3.6 of the Scrummage platform, this page is not to be confused with the Account Settings page. Account Settings is for managing users of the Scrummage platform itself, identities are an entirely optional feature, where if rows are present, the information within can be used when executing tasks.
This is the main page, depicting a table with a faux identity created for documentation purposes:
Identities can be created one of three ways:
1. Individual creation (Use the "Create Identity" function.)
2. Bulk upload of identities (Use the "Bulk Upload" function.)
3. If you have an IDM system in place, you are welcome to onboard straight to the Scrummage database, under the
org_identities table. This will help streamline and maintain your list of identities effectively.
We welcome and encourage you to contribute to the Scrummage project through the creation of new plugins. If you are interested please refer to the plugin development guide here, this will give you a run-through of how to develop a Scrummage plugin, using the custom libraries provided.
List of Current Monthly Sponsors
Become a Sponsor Now!
- Hakin9 is a monthly magazine dedicated to hacking and cybersecurity. In every edition, we try to focus on different approaches to show various techniques - defensive and offensive. This knowledge will help you understand how most popular attacks are performed and how to protect your data from them. Our tutorials, case studies and online courses will prepare you for the upcoming, potential threats in the cyber security world. We collaborate with many individuals and universities and public institutions, but also with companies such as Xento Systems, CATO Networks, EY, CIPHER Intelligence LAB, redBorder, TSG, and others.
- Blog2022.12.13What are the Common Security Weaknesses of Cloud Based Networks?
- Blog2022.10.12Vulnerability management with Wazuh open source XDR
- Blog2022.08.29Deception Technologies: Improving Incident Detection and Response by Alex Vakulov
- Blog2022.08.25Exploring the Heightened Importance of Cybersecurity in Mobile App Development by Jeff Kalwerisky
I would like to ask when is the downloadable link of that tool so I can review and possibly test it? Thanks. It will be valuable, I just didn’t see it in the article.
The installation guide is on the wiki page https://github.com/matamorphosis/Scrummage/wiki
There are also other information that you can read about!
hmm.. just have some issues with the python_requirements.txt file
└──╼ #pip install python_requirements.txt
Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://pypi.org/simple/python-requirements-txt/
Probably it was deleted or changed somewhere else. Without this file it’s not possible to run and test the tool unfortunately. Any ides for that file where could be?
Without some screenshots it might be a little difficult to give you an answer. I do have some ideas. For example you Run pip install -r requirements.txt (Python 2), or pip3 install -r requirements.txt (Python 3)
But you could try and open the request on the GitHub page, so the creator of the tool could go through the data. Maybe there is a mistake.
Hope this will help you!