
Dev Story - Session Staking Portal
/AerilymBuilding the Session Token Staking Portal end to end - React frontend, Node backend and smart contracts - lessons from my first full stack Web3 project.
The Session Token Staking Portal was the first major full stack Web3 project I built from the ground up. The project consists of three main components:
- The smart contracts
- The fontend web app
- The backend
Smart Contracts
Of the three components, I had very little to do with the smart contracts, my main contributions here were minor quality of life improvements and small bug fixes that came from my use of the contracts in the other two components.
Staking Portal
The staking portal was an enormous undertaking, it dominated my time for almost a year and was my main area of focus in the Session Token project.
Wallet Connection
The first early versions of the staking portal used Wallet Connect to facilitate user wallet connection to the portal. This worked fine, but a few areas of concern popped up through the early testnet phase.
Connecting a wallet to the website is a key first step in the onboarding process, and with Wallet Connect it was a step we had little control over. Wallet Connect started pushing email signups and changing its UI, meaning if we ever wanted to update it (which in Web3 you really need to be able to do as things change fast) we would need to accept whatever they wanted to show our users. Considering the relative simplicity of a wallet connection, I thought it was worth exploring what it took to build this myself. That's when Web3Sheet was born. I ended up building Web3Sheet myself as my own side project and it solved a lot of our concerns:
- Control. The initial user experience is dead simple, you control the onboarding.
- Headless. You can make it look however you want and control what features you want to use. It can also fit within your styling system as you supply the components it uses.
- Privacy. It's open source and completely client-side, no network requests to a home server, no analytics, its just a wallet connection package.
- Functionality. It fully integrates the wallet connection spec, the metamask SDK and even Wallet Connect, so it has all the functionality of Wallet Connect, without needing to use Wallet Connect.
Staking Backend
The staking backend was a treat. It underwent an entire ground-up rebuild during the staking portal testnet phase and required two rewrites of the Arbitrum event processing system, before the staking portal even got to version one.
The first iteration of the backend suffered from an inevitable fatal flaw, it simply couldn't keep up. The Arbitrum One network operates at a speed of 4 blocks per second, that is blazingly fast, so fast that during the Christmas break it reached its breaking point.
The initial system was designed to fetch Arbitrum event data if the last time it finished fetching data was more that 30 seconds ago. That seems fine on paper, but what happens if it takes more than 30 seconds to fetch and process the data? And what happens when the average (or every) time was more than 30 seconds?
30 seconds on Arbitrum is 120 blocks, that's 120 blocks worth of event subscriptions to fetch every 30 seconds. There were limitations to calling a massive list of addresses and a massive list of event topics so we ended up needing to chunk the requests into 6 separate requests. Initially this was fine, but something in the Arbirum RPC Node's processing code meant these requests were taking a while to process, not a crazy amount of time, but just enough that once the address list was long enough the sum of all the requests took longer than 30 seconds, that's less than one block processed per block created, a delay that would cascade into the backend becoming further and further behind real-time and lead to its eventual death loop.
In it's initial design the backend would fetch Arbitrum events every 30 seconds, with the large number of different event topics and contracts, the backend eventually overwhelmed the Arbitrum node it was using as well as overwhelming itself with the amount of requests it needed to do.
The first rewrite removed this event fetching system and replaced it with an asynchronous event subscription system, we seemed to be stumbling into untouched territory as I had to fix some of web3.py's event topic creation code to support this, but this had major improvements over the initial design as the backend could just wait to be informed of events as they happened. As the test network grew, these subscriptions became a problem, needing to subscribe to every new multi-contributor smart contract deployed presented another challenge, eventually we would run out of subscription slots, or end up in a death loop as before with the number of requests.
A second rewrite brought things back to basics, scrapping the complex and fine-grained subscription system and replacing it with something simple, stupidly simple, just a single subscription. We subscribed to every block. Every 250ms the Arbitrum Node would inform the backend of a new block, the backend would queue up a few blocks of data then just fetch the logs for all the contracts it was interested in for the new blocks. This ended up being significantly more performant and easier to code than using a bunch of complex topic filters and subscription handlers.
This was a very interesting journey, and the key takeaway for me was that sometimes the stupidly simple solution is the best one. Just because something exists to minimise the data you churn though, sometimes its easier (and better) to just churn through the data yourself.