Cache Storage and Invalidation

The browser has an inherent storage constraint so we designed our solution around efficiently managing cache storage.

Optimizing cache storage

We opted to serialize gRPC responses into array buffers prior to caching. This decision was driven by the need to conserve storage space, as array buffers are more space-efficient. The serialization process required us to also implement a corresponding deserialization mechanism. To maintain type safety and ensure accurate reconstruction of the original data structures, we associated each cache entry with a specific deserializer function reflective of its gRPC message type.

Cache invalidation strategy

There are two parts to the invalidation:

Expiration timer: This allows for precise control over cache entries, enabling developers to set expiration times based on the expected update rate of the data. Developers can configure a shorter expiration time for data that changes frequently.

Cost-aware algorithm: We introduced an algorithm that evaluates the "cost" of cache entries, which is a function of response size and access frequency. Large, infrequently accessed entries are prioritized for invalidation, optimizing the usage of the cache's limited space. To support our cost-aware algorithm, we needed to estimate the size of the responses in the cache. We designed a formula that calculates the expected byte size of an array buffer assuming a normal distribution of response sizes. And to enable quick retrieval and removal of the most costly cache entries, we implemented a max heap. This allows our algorithm to efficiently sort cache entries by their calculated cost.