Crawling the network
We want to make the Stellar network more tangible by making it visible.
To see how many nodes there are, where they are located, and how the network is growing and performing.
Since Stellar is a decentralised network, there is no easy way to get this information.
So we created a crawler that puts itself on the Stellar network and starts discovering nodes.
This works as follows:
Start with a list of known nodes.
Right now this means 'use the result of our last run', but initially we started with a list of known nodes that comes with the example configuration of stellar-core.
Connect to a node on the list and attempt to perform a basic 'hello' handshake.
If they respond we consider the node active and store it.
Ask that active node to give us a list of its known peers.
If the response contains nodes that we don't know yet, we put them on the crawler's to-do list to be discovered and asked for peers too.
- Listen to the node traffic to gather it's quorumset information
Where we go from here
As you can no doubt guess, all of this is very much work in progress.
We have a ton of ideas on statistics we want to gather, data we want to make available and code we want to contribute.
On our current list of ideas:
- Open-source our crawler
- Track additional statistics for each node that would help to see if that node is reliable (uptime,, consensus protocol actions, ...)
- Make a quorumset configurator for your own validator
- Track our statistics over a long period of time to show network progress
- Provide a search function for nodes (find by public key, facet search on various fields)
- Provide an API for all the above so others can build on top of it