Planetscape
Planetscape
Not logged in

Planetscape

Desktop app to map network-related data (like traceroutes for all active connections) to a world map.

Few years ago I've used and contribted a few patches to GeoXPlanet app.
I liked the idea, although implementation proved to be quite resource-hungry and not flexible enough.
I didn't really knew python well enough then, so I abandoned the app, but now, having some (relatively) free time on my hands, I thought I'd revive the idea and rewrite the app in a more optimal way.

Result is this project.

It's essentially a twisted-based eventloop, fetching list of active network connections (think ss (iproute2) or netstat, although I have plans to use something like libpcap-collected data to show udp connections as well), tracing routes to each, resolving each IP to a world map coordinates and rendering them as markers and arcs on xplanet image.

Features:
No threads (although, obviously there are subprocesses), fast ip->loc resolution (unlike geoxplanet, which fails to use db indexes for this), caching of everything that makes some sense (full traces to destination, reverse dns lookups, etc), clean dry codebase (thanks to twisted).

Right now it should be perfectly usable, although I'll probably change a lot of things.
GeoIP lookups require data, which can be acquired from MaxMind GeoLite City db, path to which should be specified at least on the first run:

~% py planetscape.py -1 --debug -d GeoLiteCity_20101001.zip
DEBUG:root:Globbed MaxMind db path: /home/fraggod/hatch/planetscape/GeoLiteCity_20101001.zip
DEBUG:root:Unpacking MaxMind db (to: /var/tmp/planetscape/mmdb_tmp)
DEBUG:root:Unpacked blocks: /var/tmp/planetscape/mmdb_tmp/GeoLiteCity_20101001/GeoLiteCity-Blocks.csv, blocks-loc: /var/tmp/planetscape/mmdb_tmp/GeoLiteCity_20101001/GeoLiteCity-Location.csv
DEBUG:root:Building sqlite geoip db cache
DEBUG:root:Initializing tables
...
DEBUG:root:Failed to get PTR record for 80.91.229.10
DEBUG:root:Failed to get PTR record for 130.237.188.200
DEBUG:root:Rendering XPlanet image
DEBUG:root:Exiting after first run because "oneshot" option was specified

And the result is the image like this in the root window (don't mind the low connection count on my machine atm):

MaxMind DB sucks, btw, >50% of ip ranges aren't there, including IP of this server, guess I'll look for some other ip-loc binding sources in the future.

Naturally, first run will be quite long, because GeoLiteCity has to be converted from csv. No worries, next ones will be instantenious:

~% time py planetscape.py -1 --home-lat 57 --home-lon 61 -- -output image.jpg
python planetscape.py -1 --home-lat 57 --home-lon 61 -- -output   0.57s user 0.06s system 100% cpu 0.629 total

At the moment of writing this, interface (cli only, of course) is following:

Usage: planetscape.py [options] [-- xplanet args]

Render stuff on xplanet and run some hooks

Options:
  -h, --help            show this help message and exit
  -1, --oneshot         Generate single image with a complete set of traces
                        and exit.
  -r REFRESH, --refresh=REFRESH
                        Image refresh or re-generate interval (default: 60).
  --display=DISPLAY     X display to use (default: auto-determine from env).
  -x XPLANET, --xplanet=XPLANET
                        XPlanet binary (default: xplanet).
  -n NS_TOOL, --ns-tool=NS_TOOL
                        Tool to get network connection list: ss, netstat, lsof
                        (default: ss).
  --ns-tool-binary=NS_TOOL_BINARY
                        Path to binary for selected netstat-like tool, to
                        override defaults.
  -t TRACE_TOOL, --trace-tool=TRACE_TOOL
                        Traceroute tool to use: mtr, traceroute (default:
                        mtr).
  -c TRACE_COUNT, --trace-count=TRACE_COUNT
                        Number of tracer packets to send (default: 1).
  --trace-tool-binary=TRACE_TOOL_BINARY
                        Path to binary for selected traceroute tool, to
                        override defaults.
  --trace-pool-size=TRACE_POOL_SIZE
                        Max number of traceroute subprocesses to spawn in
                        parallel (default: 20).
  --home-label=HOME_LABEL
                        Label of home-location (default: hostname).
  --home-lat=HOME_LAT   Latitude of the current location (default:
                        autodetected from external IP). Should only be
                        specified along with --home-lon.
  --home-lon=HOME_LON   Longitude of the current location (default:
                        autodetected from external IP). Should only be
                        specified along with --home-lat.
  --arc-base=ARC_BASE   File with arcs to include into rendered image
                        (default: arcs.txt).
  --marker-base=MARKER_BASE
                        File with markers to include into rendered image
                        (default: markers.txt).
  -d MAXMIND_DB, --maxmind-db=MAXMIND_DB
                        Path to new MaxMind database zip (look for it here:
                        http://www.maxmind.com/app/geolitecity). May contain
                        globbing wildcards, (like * and ?). Must be specified
                        at the first run.
  -s SPOOL_PATH, --spool-path=SPOOL_PATH
                        Path for various temporary and cache data. Dont have
                        to be persistent, but it helps performance-wise.
  --discard-cache       Invalidate trace/lookup caches on start.
  --cache-obsoletion=CACHE_OBSOLETION
                        Time, after which cache entry considered obsolete
                        (default: 3600).
  --cache-max-size=CACHE_MAX_SIZE
                        Max number of cached objects (traces, ns lookups) to
                        keep (default: 50000).
  --debug               Give extra info on whats going on.

Requirements:

Guess I'll get back to update this page once I'll get more relevant (at least to me) functionality.


Attachments: