Marcus Guttenplan | Socket.IO Network Scanner

front end dev, back end dev, design

Socket.IO Network Scanner

Building a Web Socket based Network Sniffer

Sniffing over Websockets

Setting up an express server with socket.io clients that will be used to execute nmap scans before being visualized with d3.js. Check out the live experiment.

npm i --save express socket.io node-nmap

aircrack-ng with socket.io by Marcus Guttenplan

Express + Socket.IO

// server.js
var io = require('socket.io')(server);
var express = require('express');
var app = express();
var server = require('http').createServer(app);

//redirect / to our public dir
app.use("/", express.static(__dirname + '/public/'));

// client connect
io.on('connection', function(client) {
    console.log('Client connected...');
    io.emit('clientCounter', io.engine.clientsCount);
});

//start our web server and socket.io server listening
server.listen(3000, function(){
    console.log('listening on *:3000');
});

This does absolutely nothing, except to emit a count of connected clients whenever something connects. The count can be displayed with minimal HTML, as long as it imports socket.io and listens for emitted events.

<!DOCTYPE html>
<html>
<head>
    <title>//</title>
    <meta charset="utf-8">
    <link type="text/css" rel="stylesheet" href="/css/main.css">
</head>
<body>

    <h1>Socket Scanner</h1>

    <div class="socket-counter counter">
        <p>
            <span class="count" id="clientCount">0</span> Sockets Open;<br><br>
        </p>
    </div>


    <!-- jQuery -->
    <script src="js/vendor/jquery-3.1.1.min.js"></script>

    <!-- Plugins -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.4/socket.io.js"></script>

    <!-- inline jQuery -->
    <script>
        $(document).ready(function() {});
    </script>

    <!-- Socket + d3.js -->
    <script>
        var socket = io.connect();
        var $clientCount = document.getElementById("clientCount");

         socket.on('clientCounter', function(data){
             $clientCount.innerHTML = data;
         });
    </script>
</body>
</html>

nmap

Inside of socket's io.on('connection'), which handles what happens when clients connect to the websocket, add a 'scan' function:

var nmap = require('node-nmap');

client.on('scan', function(data) {
    var scan = new nmap.OsAndPortScan("192.168.1.100-200");
    console.log("start scan");

    scan.on('complete', function(data){
        // console.log(data);
        console.log("total scan time" + scan.scanTime);
        //send a message to ALL connected clients
         io.emit('scanUpdate', data);
    });
    scan.on('error', function(error){
      console.log(error);
    });

    scan.startScan(); //processes entire queue
});

Executes a scan when the scan button is clicked, emitted from index.html via socket.emit('scan'); and received by server.js via client.on('scan', function(data) { ... });. The server responds by emitting io.emit('scanUpdate', data); which passes the results of the nmap scan back. Unfortunately, nmap requires sudo priveleges to be able to access the network interfaces, so the server has to be started with sude node server.js. This is not at all ideal for anything that will ever run in a production environment exposed to the public internet, but over a local network the risk is a bit more bearable.

The entirety of server.js now looks like :

// server.js
var express = require('express');
var app = express();
var server = require('http').createServer(app);
var io = require('socket.io')(server);
var nmap = require('node-nmap');

app.use("/", express.static(__dirname + '/public/'));

// socket client connect
io.on('connection', function(client) {
    console.log('Client connected...');

    io.emit('clientCounter', io.engine.clientsCount);

    client.on('scan', function(data) {
        // create new nmap scan isntance
        var scan = new nmap.OsAndPortScan("192.168.1.100-200");
        console.log("start scan");

        scan.on('complete', function(data){
            console.log("total scan time" + scan.scanTime);
            //emit a message to ALL connected clients with scan data
             io.emit('scanUpdate', data);
        });
        scan.on('error', function(error){
          console.log(error);
        });

        scan.startScan(); //start nmap scan
    });
});

//start web server and socket.io server listening
server.listen(3000, function(){
    console.log('listening on *:3000');
});

This is great, and handles the basics of passing data back and forth over a web socket using a front end button. But what else can be done with this?

Airodump

In my master's thesis research at art center, a friend built a useful node parser to get useful environmental wifi data from xml into a d3js front end. The constraints required a public API that the data would be POSTed to before being grabbed by the client for the visualization. Adapting this to use socket.io, we can remove the need for a public api and emit the data to anyone connected. Because it would also be sending information over web sockets instead of creating TCP traffic, this could be useful for actual engagements. This requires installing additional software, aircrack-ng. Gettable via brew install aircrack-ng && airodump-ng-oui-update on MacOS or with:

apt-get -y install libssl-dev libnl-3-dev libnl-genl-3-dev ethtool
wget http://download.aircrack-ng.org/aircrack-ng-1.2-rc3.tar.gz
tar -zxvf aircrack-ng-1.2-rc3.tar.gz
cd aircrack-ng-1.2-rc3
sudo make
sudo make install
sudo airodump-ng-oui-update

Airodump uses a promiscuous interface to sniff requests over the air, so this will also require a USB wifi dongle that is capable of being put into monitor mode. My dongle isn't compatible with modern Apple, so I'm running this server from a raspberry pi on the network.

We also need to add some additional functions to server.js that will handle spawning airodump on an interface, parsing the xml, and passing back a useful JSON object:

// airodump output json
{
  name: "probe",
  startTime: "",
  total: 0,
  unassociatedTotal: 0,
  networks (children): [
    {
      name: "essid",
      essid: "essid",
      rssi: rssi,
      packetCount: packets.total,
      clients (children): [
        {
          name: "clientMac",
          rssi: rssi,
          packetCount: packets.total,
          manufacturer: "Apple Inc"
        }
      ]
    }
  ],
  probes : [
    {
      name: bssid,
      probes: [
        "Network", "Network", "Network"
      ]
    }
  ]
}

This defines an arbitrary name for the object, as d3js uses this pattern quite often. It also grabs the start time of the scan, creates totals for found networks and unconnected probing clients. d3js also expects "children" keys when creating hierarchies. For the "children" networks, this collects the wifi network's name (essid), its signal strength (rssi), total sniffed packets, and any connected clients. For connected clients, this defines MAC address, signal strength, packet count, and manufacturer (via MAC address). Below the children networks, this object stores information about devices that have been sniffed but are not associated with any of the wifi networks found. This sniffing can reveal messages from the device broadcasting searches for past wifi networks, looking for anything to connect to. The object for these unassociated devices contains only its BSSID (MAC) and an array of those broadcast past networks, if there are any.

Ideally, these are the things we want access to, but there's so much more to grab. Run airodump to see a large table with lots of information. Some of it is harder to find in the xml output than others, but it looks like most of it is there.

npm i --save fs, watch, request, child_process, path, xml2js

Updating Server

In bite-sized chunks:

Update requirements and add some config objects

var express = require('express');
var app = express();
var server = require('http').createServer(app);
var io = require('socket.io')(server);
var nmap = require('node-nmap');
var scanner = require('node-wifi-scanner');
var fs = require('fs');
var watch = require('watch');
var request = require('request');
var parser = require('xml2json');
var spawn = require('child_process').spawn;
var isOnline = require('is-online');
var path = require('path');
var x2j = require( 'xml2js' );

var config = {
    interface: 'wlan1',
    dumpName: 'dump',
};

init() for spawning a child process that runs airodump

function init() {
  console.log('Attempt to execute airodump-ng');
  // parseData('./data/dump-08.kismet.netxml');

  var cmd = spawn('airodump-ng', [
    '-w ' + config.dumpName,
    config.interface
  ], {cwd: 'data'});

  cmd.stdout.on('data', function (data) {
    //console.log('stdout: ' + data);
  });

  cmd.stderr.on('data', function (data) {
    //console.log('stderr: ' + data);
  });

  cmd.on('close', function (code) {
      parseData('./data/dump-08.kismet.netxml');
    console.log('child process exited with code ' + code + '. Make sure your wifi device is set to monitor mode.');
  });

  // TODO: Start this when cmd is connected instead of on a timeout
  setTimeout(function() {startWatching();}, 10000);
}

A watcher to check for changes to the airodump output files

function startWatching() {
  console.log('Watching for changes to airodump data');

  // Watch for file changes in data folder
  watch.createMonitor('./data', function (monitor) {
    monitor.on('changed', function (file, curr, prev) {
      // Filter out netxml files
      if (path.extname(file) === '.netxml') {
        parseData(file);
      }
    });
  });
}

A parser to turn the output xml into semi-usable json

function parseData(file) {
  console.log('Parsing data for: ' + file);

  try {
    var xml = fs.readFileSync(file);
    var p = new x2j.Parser({strict:false});

    p.parseString(xml, function( err, result ) {
        var cleanJson = result;
        postData(result);
    });

    isOnline(function(err, online) {
      if (err) throw err;

      if (online === true) {
        // Device is online
        console.log('Device is online');
        postData(data);
      } else {
      }
    });
  } catch(e) {
    console.log('There was an error parsing your xml');
    console.log(e);
  }

}

Check for duplicate results and remove

function dupeCheck(arr, prop) {
    var new_arr = [];
    var lookup = {};
    for (var i in arr) {
        lookup[arr[i][prop]] = arr[i];
    }
    for (i in lookup) {
        new_arr.push(lookup[i]);
    }
    // console.log(new_arr);
    return new_arr;
}

Pass cleaned data over socket

function postData(json) {
  try {
        io.emit('airodump', cleanData(json));
  } catch (e) {
    console.log('There was an error in the request to the API');
    console.log(e);
  }
}

Clean the parsed data and create a nice json object

function cleanData(data) {
    console.log("cleaning JSON");
    var cleanJson = {};
    var startTime = data["DETECTION-RUN"]["$"]["START-TIME"];
    var networkArray = data["DETECTION-RUN"]["WIRELESS-NETWORK"]

    cleanJson.name = "probe";
    cleanJson.start = startTime;
    cleanJson.children = [];
    cleanJson.probes =[];

    // iterate through scanned networks
    for (var i=0; i < networkArray.length; i++) {
        var curr = networkArray[i]; // current network
        var currCli = curr["WIRELESS-CLIENT"]; // current clients
        var rssi = curr["SNR-INFO"][0]["LAST_SIGNAL_RSSI"][0]; // current signal strength
        var packetCount = curr["PACKETS"][0]["TOTAL"][0]; // current packet count

        // HACK to fix d3 "undefined" error
        // Check if SSID and then ESSID (plain network name) exists
        if (curr["SSID"]) {
            if(curr["SSID"][0]){
                if(curr["SSID"][0]["ESSID"]) {
                    if(curr["SSID"][0]["ESSID"][0]) {
                         var essid = curr["SSID"][0]["ESSID"][0]["_"]; // current ESSID
                    }
                }
            }
        }

        var networksobj = {};
        var probeobj = {};

        if (essid) {
            networksobj.name = essid;
        }
        if (rssi) {
            networksobj.rssi = rssi;
        }

        networksobj.packetCount = packetCount;

        // If network has clients
        if (currCli) {
            networksobj.children = [];
            var cliMac = currCli[0]["CLIENT-MAC"][0]; // client MAC
            var cliMan = currCli[0]["CLIENT-MANUF"][0]; // client manufacturer
            var cliRssi = currCli[0]["SNR-INFO"][0]["LAST_SIGNAL_RSSI"][0]; // client signal strength
            var packetCount = currCli[0]["PACKETS"][0]["TOTAL"][0]; // client total packet count
            var clientobj = {}
            clientobj.name = cliMac;
            clientobj.mac = cliMac;
            clientobj.manufacturer = cliMan;
            clientobj.rssi = cliRssi;
            clientobj.packetCount = packetCount;

            networksobj.children.push(clientobj);
        }

        // If client is sending probes
        if (curr["$"]["TYPE"] === "probe") {
            // If client is broadcasting past networks
            if (curr["WIRELESS-CLIENT"]) {
                // If past networks have names
                if(curr["WIRELESS-CLIENT"][0]["SSID"][0]["SSID"]) {
                    probeobj.probes = [];
                    // Iterate through past networks
                    for (var x=0; x<curr["WIRELESS-CLIENT"][0]["SSID"].length; x++) {
                        probeobj.name = curr["BSSID"][0]; // Probe BSSID
                        probeobj.probes.push(curr["WIRELESS-CLIENT"][0]["SSID"][x]["SSID"][0]); // Probe's past networks
                    }
                    cleanJson.probes.push(probeobj);
                }

            }
        }

        cleanJson.children.push(networksobj);
        cleanJson.children = dupeCheck(cleanJson.children, 'name'); // de-dupe by network name
        cleanJson.probes = dupeCheck(cleanJson.probes, 'name'); // de-dupe by BSSID
        cleanJson.total = cleanJson.children.length;
        cleanJson.unassociated = cleanJson.probes.length;
    }
    return cleanJson;
    // postData(cleanJson);
}

And the rest of server.js

app.use("/", express.static(__dirname + '/public/'));

// socket client connect
io.on('connection', function(client) {
    console.log('Client connected...');

    io.emit('clientCounter', io.engine.clientsCount,'utf-8');

    client.on('scan', function(data) {
      scanCount++;

        var scan = new nmap.OsAndPortScan("192.168.1.100-200");
        console.log("start scan");

        scan.on('complete', function(data){
            // console.log(data);
            console.log("total scan time" + scan.scanTime);
            //send a message to ALL connected clients
             io.emit('scanUpdate', data, 'utf-8');
        });
        scan.on('error', function(error){
          console.log(error);
        });

        scan.startScan();
    });

    client.on('dump',function(client){
        init();
    });

    client.on('quickscan', function(client){
        var scan = new nmap.QuickScan("192.168.1.100-200");
        console.log("start scan");

        scan.on('complete', function(data){
            // console.log(data);
            console.log("total scan time" + scan.scanTime);
            //send a message to ALL connected clients
             io.emit('scanUpdate', data, 'utf-8');
        });
        scan.on('error', function(error){
          console.log(error);
        });

        scan.startScan();
    });

});


//start our web server and socket.io server listening
server.listen(3000, function(){
    console.log('listening on *:3000');
    // startWatching();
    init();
});

Quickly broken down:

We specify options via a config object to make changing things easier
The init() function spawns a child process: airodump, which starts startWatching(); to watch for dumped xml files.
Dumped xml is passed to parseData(); where it is parsed and passed to cleanData();
cleanData(); iterates over the parsed data and pushes values into a much more useful JSON object, and then passes that to postData();
dupeCheck(); takes an array of javascript objects (networks and probes) and a key to check ("name"), and finds and removes duplicates objects.
postData(); sends the useful JSON to connected socket.io clients.

Good to go. If all modules and necessary software is installed, we can now run an nmap scan of a local network AND run an airodump scan for surrounding networks and devices.

HTML

With some simple HTML, we can receive and display the raw data:

<!-- public/index.html -->
<!DOCTYPE html>
<html>

<head>
    <title>//</title>
    <meta charset="utf-8">
    <link type="text/css" rel="stylesheet" href="/css/main.css">
</head>


<body>

    <h1>Socket Scanner</h1>

    <div class="scan-buttons">
        <button id="scan" onclick="dump()">AIRODUMP</button>
        <button id="scan" onclick="scan()">SCAN</button>
        <button id="quickscan" onclick="quickscan()">QUICK SCAN</button>
    </div>

    <div class="counterbucket">
        <div class="socket-counter counter">
            <p>
                <span class="count" id="clientCount">0</span> Sockets Open;<br><br>
            </p>
        </div>

        <div class="scan-counter counter">
            <p>
                <span class="count" id="scanCount">0</span> Access Points;<br><br>
            </p>
        </div>

        <div class="probe-counter counter">
            <p>
                <span class="count" id="probeCount">0</span> Unassociated Clients;<br><br>
            </p>
        </div>
    </div>

    <div id="airodump-output"></div>
    <div id="clientList"></div>
    <div id="chart"></div>

    <script src="http://d3js.org/d3.v2.min.js?2.9.3"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/socket.io/2.0.4/socket.io.js"></script>
    <script>
        var socket = io.connect();
        var $results = document.getElementById('results');
        var $airodumpOutput = document.getElementById('airodump-output');
        var $clientCount = document.getElementById("clientCount");
        var $clientList = document.getElementById("clientList");
        var $scanCount = document.getElementById("scanCount");
        var $probeCount = document.getElementById("probeCount");

        function scan(){
            $results.innerHTML = "";
           socket.emit('scan');
         }

         function quickscan(){
             $results.innerHTML = "";
           socket.emit('quickscan');
         }

         function dump(){
           socket.emit('dump');
         }

         socket.on('clientCounter', function(data){
             $clientCount.innerHTML = data;
         });

         socket.on('airodump', function(data){
            $scanCount.innerHTML = data.total;
            $probeCount.innerHTML = data.unassociated;

            console.log(data);
         });

        socket.on('scanUpdate', function(data){
             $scanCount.innerHTML = data.length;
             var htmlString = "";

             for (var i=0; i < data.length; i++) {
                 if (data[i].openPorts) {
                     for (var x=0; x<data[i].openPorts.length; x++) {
                        htmlString += '<ul class="port">' + data[i].openPorts[x].port + ',' + data[i].openPorts[x].service + '</ul>';
                     }
                 }
                 $results.insertAdjacentHTML('beforeend', '<div class="scan-result"><div class="host">' + data[i].hostname + '</div><div class="os">' + data[i].osNmap + '</div><div class="ip">' + data[i].ip + '</div><div class="ports">' + htmlString + '</div></div>');
             }
         });
    </script>
</body>

</html>

d3js

This data is just waiting to be visualized, and having built the JSON object with d3 in mind, why not throw it into a tree diagram? Add this function to the bottom index.html's script tag, below the socket functions.

function d3init(datain) {
    var diameter = document.documentElement.clientHeight;
    var viewerWidth = document.documentElement.clientWidth;
    var viewerHeight = document.documentElement.clientHeight;

   // var margin = {top: 20, right: 120, bottom: 20, left: 120},
   var margin = {top: 200, right: 20, bottom: 20, left: 20},
       width = viewerWidth,
       height = viewerHeight;



   var i = 0,
       duration = 0,
       root;

   var tree = d3.layout.tree()

       .separation(function(a, b) { return (a.parent == b.parent ? 5 : 10) / a.depth; })
       // .separation(function(a, b) { return ((a.parent == root) && (b.parent == root)) ? 3 : 1; })
       .size([360, diameter/2-80]);
       // .size([height, width - 160]);

   var diagonal = d3.svg.diagonal.radial()
       .projection(function(d) { return [d.y, d.x / 180 * Math.PI]; });

   d3.select("svg").remove();
   var svg = d3.select("#chart").append("svg")
       .attr("width", width )
       .attr("height", height )
     .append("g")
       .attr("transform", "translate(" + height / 2 + "," + width / 2 + ")");

   root = datain;
   root.x0 = height / 2;
   root.y0 = 0;

   //root.children.forEach(collapse); // start with all children collapsed
   update(root);

   function update(source) {
     // Compute the new tree layout.
     var nodes = tree.nodes(root),
         links = tree.links(nodes);

     // Normalize for fixed-depth.
     nodes.forEach(function(d) { d.y = d.depth * 300; });

     // Update the nodes…
     var node = svg.selectAll("g.node")
         .data(nodes, function(d) { return d.id || (d.id = ++i); });

     // Enter any new nodes at the parent's previous position.
     var nodeEnter = node.enter().append("g")
         .attr("class", "node")
         .attr("transform", function(d) { return "rotate(" + (d.x - 90) + ")translate(" + d.y + ")"; })
         .on("click", click);

    // Append Circles
     nodeEnter.append("circle")
         .attr("r", 1e-6)
         .style("fill", function(d) { return d._children ? "#fc533e" : "default"; });

     nodeEnter.append("text")
         .attr("x", 10)
         .attr("dy", ".35em")
         .attr("text-anchor", "start")
         .attr("transform", function(d) { return d.x < 180 ? "translate(0)" : "rotate(180)translate(-" + (Math.min(d.name.length, 10) * 8.5) + ")"; })
         .text(function(d) {
               if (d.name) {
                   if (d.name.length > 10) {
                       return d.name.substring(0,10)+'...';
                   }
                    else {
                        return d.name;
                    }
               } else {
                   return "unknown";
               }

         })
         .style("fill-opacity", 1e-6);

     // Transition nodes to their new position.
     var nodeUpdate = node.transition()
         .duration(duration)
         .attr("transform", function(d) { return "rotate(" + (d.x - 90) + ")translate(" + d.y + ")"; })

     nodeUpdate.select("circle")
         .attr("r", function(d) { return d.packetCount ? Math.min(d.packetCount, 4.5) : 4.5; })
         .style("fill", function(d) { return d._children ? "#fc533e" : "default"; });

     nodeUpdate.select("text")
         .style("fill-opacity", 1)

     // TODO: appropriate transform
     var nodeExit = node.exit()
         .remove();

     nodeExit.select("circle")
         .attr("r", 1e-6);

     nodeExit.select("text")
         .style("fill-opacity", 1e-6);

     // Update the links…
     var link = svg.selectAll("path.link")
         .data(links, function(d) { return d.target.id; });

     // Enter any new links at the parent's previous position.
     link.enter().insert("path", "g")
         .attr("class", "link")
         .attr("d", function(d) {
           var o = {x: source.x0, y: source.y0};
           return diagonal({source: o, target: o});
         });

     // Transition links to their new position.
     link.transition()
         .duration(duration)
         .attr("d", diagonal);

     // Transition exiting nodes to the parent's new position.
     link.exit().transition()
         .duration(duration)
         .attr("d", function(d) {
           var o = {x: source.x, y: source.y};
           return diagonal({source: o, target: o});
         })
         .remove();

     // Stash the old positions for transition.
     nodes.forEach(function(d) {
       d.x0 = d.x;
       d.y0 = d.y;
     });
   }

   // Toggle children on click.
   function click(d) {
     if (d.children) {
       d._children = d.children;
       d.children = null;
     } else {
       d.children = d._children;
       d._children = null;
     }

     update(d);
   }

   // Define the zoom function for the zoomable tree
   function zoom() {
       svgGroup.attr("transform", "translate(" + d3.event.translate + ")scale(" + d3.event.scale + ")");
   }

   // define the zoomListener which calls the zoom function on the "zoom" event constrained within the scaleExtents
   var zoomListener = d3.behavior.zoom().scaleExtent([0.1, 3]).on("zoom", zoom);

   function centerNode(source) {
       scale = zoomListener.scale();
       x = -source.y0;
       y = -source.x0;
       x = x * scale + viewerWidth / 2;
       y = y * scale + viewerHeight / 2;
       d3.select('g').transition()
           .duration(duration)
           .attr("transform", "translate(" + viewerWidth / 2 + "," + viewerHeight / 2 + ")scale(" + scale + ")");
       zoomListener.scale(scale);
       zoomListener.translate([x, y]);
   }

   // Collapse nodes
   function collapse(d) {
     if (d.children) {
         d._children = d.children;
         d._children.forEach(collapse);
         d.children = null;
       }
   }

   root.children.forEach(function(child){
       collapse(child);
   });

   update(root);
   centerNode(root);
}

And update the airodump socket function to call the new d3init() function, passing the data:

socket.on('airodump', function(data){
   $scanCount.innerHTML = data.total;
   $probeCount.innerHTML = data.unassociated;

   d3init(data);
});

And some extremely minimal scss

button {
    padding: 5px 10px;
    margin: 0 5px;
    background-color: $black;
    color: #fff;
}

.node circle {
    fill: $gray;
    stroke: $gray;
    stroke-width: 1.5px;
}

.node {
    font: 10px sans-serif;
}

.link {
    fill: none;
    stroke: #ccc;
    stroke-width: 1.5px;
}

.counterbucket {
    position: absolute;
    right: 0;
    top: 0;
    height: 100%;
    display: flex;
    flex-direction: column;
    justify-content: flex-end;
    padding: 25px;
    .counter {
        text-align: center;

        .count {
            font-size: 5rem;
            font-weight: bold;
            display: block;
        }
    }
}

#chart {
    position: absolute;
    top: 0;
    right: 0;
    bottom: 0;
    left: 0;
    height: 100vh;
    width: 100vw;
    z-index: -1;
}