While working through the potato chat BLOBs, I was introduced to the varint concept. Effectively, a varint is a flexible way to communicate positive integers through one or more byte of data, dynamically.
Instead of an application always requiring an integer use *one* byte of data to convey a number (and thus being limited from 0-255), or the application always requiring some larger number of bytes to convey a larger span of integers, a varint can be used.
A varint, or “variable integer”, allows your program to convey numbers from zero to… a lot more than 255 by way of dynamically allocating bytes as needed. Much in the same way that a signed integer uses the most significant byte (MSB) to declare if an integer is negative or not, a varint uses the MSB to declare if another byte needs to be involved in the calculation of the number or not. How it works in practice is this:
Given the initial bytes of 0x9403, we must first break the bytes down to binary. Since 0x93 has the MSB set as a 1, it declares that another byte is necessary to complete the integer. Because that MSB is being used as a flag rather than part of the integer, we only count the other seven bits as part of the integer. The next byte, 0x03, has a MSB of 0, saying no more bytes are necessary for the calculation of the integer.
We next drop the MSB from each of the involved bytes, order them little endian, smash all the bits together and re-weight them accordingly. This leaves us with a bit string of 00 0001 1001 0100. Applying normal bit values and adding them together, we get 256 + 128 + 16 + 4, which equates to 404.
Now – I could have undertaken this tedious process manually, or asked some LLM to do the math for me (they get it wrong sometimes… better know how to verify their math). Instead, I briefly googled around for a calculator until I remembered that I can fumble around clumsily in python and sometimes get things to work right! So, that’s what I did. I put together a small GUI app that I can type a varint into and get an integer out of. This has proven useful during my time manually benchmarking the potato chat BLOBs, so I wanted to make it available for anyone else who might want it. If you find it useful, please let me know! If you break it, also let me know, I learn by breaking things and I am far from an expert in this realm.
If you want to check it out, go grab the script here:
Leave a Reply