.svg)
Invoking any generative AI model can be a costly endeavor. Each call to complete a prompt or generate an image can cost a few cents which quickly compounds when dealing with the larger datasets of most spreadsheets. Add in the fact that most spreadsheets refresh their cell values on a fairly regular interval, or in response to several events in the spreadsheet, and you can quickly hit the limits of any budget.
In our effort to make SheetGPT the most cost-effective, performance, and efficient way to utilize GPT in Google Sheets, we have implemented several layers of performance enhancements while still retaining a native spreadsheet experience. In general, you should not have to think about how to optimize your GPT usage – we've done the heavy lifting for you.
Value caching
By default, every value that is returned by a GPT function is cached on two levels – the local cache and a remote cache.
The local cache is provided by Google Sheets and is the most performant. However, it is limited by Google to have a max storage time of six hours and up to 1,000 items. It also gets wiped on every cell movement, so even adding a row to a sheet can cause an unrelated cell to re-evaluate. If the GPT function call is not found in the local cache, SheetGPT looks in its remote cache.
The remote cache stores values for up to 3 months and does not clear on unrelated sheet adjustments. While it is not as performant as the local cache, it is still quite quick and avoids hitting the OpenAI API again which often saves hundreds of tokens per execution.
All identical GPT function calls with the same arguments will return the same cached value and will not cause a new generation to occur. Even when you see the cell re-evaluating, it is only hitting one of our caches.
Image caching
In addition to returning the same image URL value from identical GPTIMAGE calls, SheetGPT also stores the resulting image in its own, image-optimized, CDN-enabled location. This ensures we are not relying on AI infrastructure which as very short expiration periods on generated assets and, in general, is less reliable.
Forcing re-generation
There may be times when you want to avoid hitting the cache and force a re-generation of the given prompt. You can do with by setting the cache argument to false in the corresponding GPT function call. For instance, with "GPT":
=GPT("Who was the best US president?",,,,false)
Resources
You may find these resources helpful as well: