TapAndMap

Welcome to the TapAndMap family!   


TapAndMap is visually described in this YouTube video:

Basically, with a Raspberry Pi and a couple of USB ethernet dongles, you can tap any connection (say between your computer and your router, or even between your router and your modem), and see all of your network traffic on Google Maps!  You can also use a single USB ethernet dongle if you have a network sniffer, a spanning port, or an old school hub.  


To install TapAndMap, first download the necessary files from github at:

http://github.com/johnbebo/TapAndMap

After you have the files, here's how you install your TapAndMap (instructions are current as of 28 Dec 2017):

  
-) download 2017-11-29-raspbian-stretch-lite.zip and unzip it to 2017-11-29-raspbian-stretch-lite.img
-) copy that to your SD card (I did dd if=2017-11-29-raspbian-stretch-lite.img of=/dev/sdb, but BE CAREFUL to know what you're doing so that you don't nuke your main OS or something)
-) boot into the Raspberry Pi with a keyboard and mouse (username/password is pi/raspberry)
-) enable ssh on the pi by doing this:  'touch /boot/ssh' and reboot, and maybe change the pi password if you like (passwd pi)
-) set up your networking:
  If you want it, I included an 'interfaces.gz' file.  I just copied the 'interfaces' file to /etc/network/interfaces 
  update /etc/resolv.conf for your DNS server of choice
  reboot
  kill dhcpcd just in case it gets in the way
      update-rc.d dhcpcd disable
      service dhcpcd stop
-) update:
      apt-get update -y
      apt-get dist-upgrade -y
-) install these packages
      apt-get install tcpdump -y                # needed to sniff and in case you need to troubleshoot
      apt-get install screen -y                   # needed to ssh into the pi, run a script, and leave it running by 'ctrl-a d' out of that screen, and 'screen -x' back into it if necessary
      apt-get install python-pcapy -y # needed to parse packets
      apt-get install bridge-utils -y           # needed to make a bridge
-) install MaxMind GeoIP database and test it out
      apt-get install geoip-bin geoip-database -y
      Get the MaxMind GeoLiteCity database by going to https://dev.maxmind.com/geoip/legacy/geolite/ and download the binary/gzip file. 
      extract the GeoLiteCity.dat.gz file with gunzip GeoLiteCity.dat.gz
      mkdir /usr/local/share/GeoIP
      mv GeoLiteCity.dat /usr/local/share/GeoIP/
      test that your GeoIP lookup works with 'geoiplookup -f /usr/local/share/GeoIP/GeoLiteCity.dat x.x.x.x'  where x.x.x.x is a public IP address that you'd like to look up
-) install apache
      apt-get install apache2 -y
      Test it by going to the IP address with a browser.
      change its home directory from /var/www/html to /var/www
            edit the file /etc/apache2/sites-enabled/000-default.conf to make the 'DocumentRoot /var/www' (it was /var/www/html on the newest version)
        service apache2 restart
        Test it by going to the IP address... you should see an index listing now
-) Transfer over the TapAndMap website files
      copy the var-www-files.tar.gz to /tmp with something like this:   scp -p var-www-files.tar.gz  pi@10.50.60.18:/tmp/ (the '-p' will keep file permissions... that's always a good idea, as is scping them to /tmp/)
      extract in the pi's /tmp/ directory with something like this:  tar -xzvf var-www-files.tar.gz
      cd /tmp/var/www/
      mv *  /var/www/
      Now when you open the site with the browser, you'll see the glorius TapAndMap page!
-) You'll need to create a bridged interface to sniff from, and add eth1 and maybe eth2 to it.  If you have a spanning port or a hub and only one USB dongle, then just add eth1.  If you have two USB dongles, and use your device to bridge between both of those dongles, then add eth2 as well
                  Thus, your /etc/rc.local file should have either this:
                  ifconfig eth1 up
                  brctl addbr br0
                  brctl addif br0 eth1
                  ifconfig br0 up
               
                  or it should look like this:
                  ifconfig eth1 up
                  ifconfig eth2 up
                  brctl addbr br0
                  brctl addif br0 eth1 eth2
                  ifconfig br0 up
  reboot
                  you should see a br0 interface.   This is important, as this is the interface that you'll sniff from.  It can either be connected to one USB-to-Ethernet dongle (for a spanning port), or bridge two USB-to-Ethernet dongles
                  to make sure that it works, run 'tcpdump -i br0' and you should see packets flowing.  If this doesn't work, then troubleshoot until you see packets flowing
-) Now we need to place two files in the /usr/lib/cgi-bin/ directory to allow for the GUI to reset the map or textual data with the user mouse-click
        scp  the usr-lib-cgi-bin-files.tar.gz file to /tmp/
        cd /tmp/
        extract it in the /tmp/ directory with 'tar -xvzf usr-lib-cgi-bin-files.tar.gz'
      cd /tmp/usr/lib/cgi-bin/
      mv ResetScreen.sh /usr/lib/cgi-bin/
      mv ResetText.sh /usr/lib/cgi-bin/
      When you first go to the TapAndMap homepage in a browser, and click on a "Clear Map" or "Clear Text" button, you will get a 404 page Not Found error.  This is because apache has forbidden access to the cgi-bin files. 
     
        First, you'll need to allow access to the cgi-bin files
            edit the /etc/apache2/sites-enabled/000-default.conf file to uncomment the last line (#Include conf-available/serve-cgi-bin.conf)
            Then modify that file that you've uncommented the line to point to (/etc/apache2/conf-available/serve-cgi-bin.conf).  Mine looked like this:
                        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
                        <Directory "/usr/lib/cgi-bin">
                                  AddHandler cgi-script .sh
                                  AllowOverride All
                                  Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                                  Require all granted
                        </Directory>
              Then modify the /etc/apache2/conf-enabled/security.conf file for access to the directory.  I added these lines:
                  ScriptAlias /cgi-bin /usr/lib/cgi-bin/
                    <Directory /usr/lib/cgi-bin>
                            AllowOverride All
                            Require all granted
                  </Directory>
        Now, if you do a 'service apache2 restart' and click on the "Clear Map" or "Clear Text" button, you'll see the script.  That means that it's accessable.  But if you notice, it says "403 Forbidden"  To make it run when clicked,
              mv /etc/apache2/mods-available/cgi* /etc/apache2/mods-enabled/
              This will copy the cgi mods from available to enabled, and allow you to execute the script on the buttons.   This won't take effect until you do another 'service apache2 restart.'  You may also have to clear the data in your history to make this work.  The history has a way of messing up the data.
-) That should be it!  Let's test it! 
    cd /var/www
    screen  # and then space... this is so you can run the Run_TapAndMap_V1.3.1.py python script of TapAndMap and then leave it run and exit out of the terminal (yet still come back to it... way better than running it in the background with &).
  You should see something like this:
  "IPs to ignore are: 10. 169.254. 192.168. 172.16. 0. 223. 224. 225. 226. 227. 228. 229. 230. 231. 232. 233. 234. 235. 236. 237. 238. 239. 240. 255. 8.8.8.8 209.85. 172.217. 64.233. 216.58. 173.194. 74.125. 208.67.
"
  'Ctrl-a d' to exit that screen, and you'll be back at your normal terminal
  'ping www.china.cn -c 1'  to send a ping to china
  refresh your browser, and you should see the ping graphically going to China, and textually see it on the right pane
  from your terminal, 'screen -x' will bring you back into your python script so you can see how it's doing.   'ctrl-a d' will detatch from that screen window again. 
 
-) there is a /var/www/TapAndMap.conf file that will allow you to change your home lat/long, timezone, and how often the log will split.   You might want to back this up before you mess with it. 
  Connection logs are stored under /var/www/logs/   anything in this directory can be deleted if you don't want it any longer
  JSON Files (for maps) are stored under /var/www/JSONFiles/   anything in this directory can be deleted if you don't want it any longer
  map htmp files are stored under /var/www/maps/  anything in this directory can be deleted if you don't want it any longer
  There are a couple of script for your convienience and testing.   Try out /var/www/PingWorld.py and /var/www/ScanWorldWithUDP.py if you need to generate test traffic.
  If you run into problems, like your browser not refreshing when told to do so, first try deleting the files from the browser history...  This will usually fix it.  This is a known bug (known that it exists, cause/fix is still unknown).

That's it!  You're now mapping your network traffic!   

1 Sep 2018 update:

Wait a second.... Google Maps has started enforcing their Google Maps Javascript API key, so the above WAS working fine, and will still work, but now you'll get an annoying dark map (watermarked map) and a pop-up that asks if you really own the website.  

To fix that, I updated the index_template.html and map_template.html line which read:

<script src="http://maps.googleapis.com/maps/api/js?libraries=geometry&sensor=false"></script>

to be:

<script src="https://maps.googleapis.com/maps/api/js?libraries=geometry&key=YOUR_KEY"></script>

How do you get a key you ask?  

Go to:

https://cloud.google.com/maps-platform/#get-started

and it will say "Welcome to Google Maps"



Then "Get Started" and "Create a Project"


Then enable a GoogleMaps platform, and create a billing account.  

After you put in your credit card, you'll get an EnableAppsAPI screen -> Next


Then you'll get an API Key.  Put that key in your line, and you'll not be annoyed by that dark screen.   I think you get like 10,000 loads before they charge you.  If I get charged, I'll update this blog.


If you like what you see, and you're feeling generous, feel free to donate a few fractions of a bitcoin to: 1Pq1pwxauSj1zQQLLQA4Suc9azVfFYU2dX

(That QR code is below in case you want to take a pic with your phone)