Changelog SpiderFoot 2.10 is a big release including a CLI, four new modules, new data sources and bug fixes.
* New functionality
— After much demand, SpiderFoot now finally has a CLI! Watch this space for some video tutorials. Meanwhile, python ./sfcli.py –help is your friend.
* New Modules / Data Sources
— This release introduces five new modules and an update to sfp_sharedip to include an additional data source:
+ sfp_googlemaps: Identifies potential physical addresses and latitude/longitude coordinates.
+ sfp_clearbit: Searches Clearbit Check for names, addresses, domains and more based on lookups of e-mail addresses identified.
+ sfp_alienvault: Searches AlienVault OTX for threat intelligence related to idenified IPs, netblocks, hostnames and affiliates.
+ sfp_bitcoin: Looks for Bitcoin wallet addresses in fetched web content.
+ sfp_malcheck: Now queries Fortinet and Watchguard’s threat intelligence data feeds. Dead packetmail.net references replaced with kittenlasers.com.
+ sfp_dns: Now identifies DNS SPF records.
* Enhancements / Bug fixes
+ Added a requirements.txt file for better dependency management.
+ UI “New Scan” and “Options” screens now show a lock icon next to modules requiring API keys.
+ SpiderFoot might sometimes hang on some DNS queries, so a timeout was introduced.
+ When fetching only HEAD requests (to determine the size of a request), a timeout was missing which is now added.
+ Additional information has been added to the About box.
+ Scan IDs are now much shorter, mainly to make workin with them in the CLI easier.
+ Misc. minor bug fixes and tweaks.
+ Enhancements / Bug fixes – sfp_accounts will produce way less false positives as it now also checks for the queried username to be mentioned within the returned social media site content. – sfp_names and others now use French, German and Spanish as well as English dictionaries for better word/name detection which ultimately means less false positives. – Removed the problematic use of metapdf in favor of PyPDF2. – Removed the inclusion of BeautifulSoup and now added it as a requirement to be installed. This should address the compatibility issues many experienced on Kali Linux. – The Windows binary has been compiled to not include SSL libraries causing conflicts on older versions of Windows, addressing errors some were seeing when trying to run 2.8.0 on Windows 7. – Miscellaneous tweaks and improvements to reduce crashes/errors and detect more data.
SpiderFoot is an open source intelligence automation tool. Its goal is to automate the process of gathering intelligence about a given target, which may be an IP address, domain name, hostname or network subnet. SpiderFoot can be used offensively, i.e. as part of a black-box penetration test to gather information about the target or defensively to identify what information your organisation is freely providing for attackers to use against you.
tar xf v2.10-final.tar.gz
pip install -r requirements.txt
python sfcli.py --help
then browse to http://127.0.0.1:5001