{"id":157,"date":"2025-04-20T19:43:44","date_gmt":"2025-04-21T02:43:44","guid":{"rendered":"https:\/\/forrestcook.net\/?p=157"},"modified":"2025-04-20T19:43:44","modified_gmt":"2025-04-21T02:43:44","slug":"to-varint-or-not-to-varint","status":"publish","type":"post","link":"https:\/\/forrestcook.net\/?p=157","title":{"rendered":"To Varint or not to varint"},"content":{"rendered":"\n

While working through the potato chat BLOBs, I was introduced to the varint concept. Effectively, a varint is a flexible way to communicate positive integers through one or more byte of data, dynamically. <\/p>\n\n\n\n

Instead of an application always requiring an integer use *one* byte of data to convey a number (and thus being limited from 0-255), or the application always requiring some larger number of bytes to convey a larger span of integers, a varint can be used.<\/p>\n\n\n\n

A varint, or “variable integer”, allows your program to convey numbers from zero to… a lot more than 255 by way of dynamically allocating bytes as needed. Much in the same way that a signed integer uses the most significant byte (MSB) to declare if an integer is negative or not, a varint uses the MSB to declare if another byte needs to be involved in the calculation of the number or not. How it works in practice is this:<\/p>\n\n\n\n

\"\"<\/figure>
\n

Given the initial bytes of 0x9403, we must first break the bytes down to binary. Since 0x93 has the MSB set as a 1, it declares that another byte is necessary to complete the integer. Because that MSB is being used as a flag rather than part of the integer, we only count the other seven bits as part of the integer. The next byte, 0x03, has a MSB of 0, saying no more bytes are necessary for the calculation of the integer.<\/p>\n<\/div><\/div>\n\n\n\n

We next drop the MSB from each of the involved bytes, order them little endian, smash all the bits together and re-weight them accordingly. This leaves us with a bit string of 00 0001 1001 0100. Applying normal bit values and adding them together, we get 256 + 128 + 16 + 4, which equates to 404.<\/p>\n\n\n\n

Now – I could have undertaken this tedious process manually, or asked some LLM to do the math for me (they get it wrong sometimes… better know how to verify their math). Instead, I briefly googled around for a calculator until I remembered that I can fumble around clumsily in python and sometimes get things to work right! So, that’s what I did. I put together a small GUI app that I can type a varint into and get an integer out of. This has proven useful during my time manually benchmarking the potato chat BLOBs, so I wanted to make it available for anyone else who might want it. If you find it useful, please let me know! If you break it, also let me know, I learn by breaking things and I am far from an expert in this realm.<\/p>\n\n\n\n

If you want to check it out, go grab the script here:<\/p>\n\n\n\n

https:\/\/github.com\/Whee30\/varint_calc<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

While working through the potato chat BLOBs, I was introduced to the varint concept. Effectively, a varint is a flexible way to communicate positive integers through one or more byte of data, dynamically. Instead of an application always requiring an integer use *one* byte of data to convey a number (and thus being limited from […]<\/p>\n","protected":false},"author":2,"featured_media":158,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,4],"tags":[6,7,8,9,10,12,13,15],"class_list":["post-157","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-github","category-personal","tag-app","tag-application","tag-coding","tag-dfir","tag-digital-forensics","tag-programming","tag-python","tag-technology"],"_links":{"self":[{"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/posts\/157","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/forrestcook.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=157"}],"version-history":[{"count":0,"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/posts\/157\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/forrestcook.net\/index.php?rest_route=\/wp\/v2\/media\/158"}],"wp:attachment":[{"href":"https:\/\/forrestcook.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=157"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/forrestcook.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=157"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/forrestcook.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=157"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}