Resolving Google Gemini API Error 500 in Python with try-except Loops

While user-facing LLMs like ChatGPT have their uses, their true potential shows when integrated into programs via APIs. This approach allows data scientists to process or generate natural language directly within their code.

Google Gemini has become my go-to when I need an LLM API for one simple reason: it has a free tier. It is limited to 60 requests per minute, but for small scale projects this is more than enough.

Recently, however, I started getting a random Error: 500 responses. One time, a block of code would run perfectly, but the next time, it would throw an error. Error:500 is a server-side issue with Google as opposed to a problem with the code itself, API keys, or rate limits.

Regardless of the cause, these occasional errors were frustrating. I'm currently using Gemini API with a few-shot prompt to generate consult questions for a synthetic dataset of palliative care consultations. To do this, I'm sending dozens or hundreds of API requests in sequence. With my initial code, a single Error: 500 would crash the program.

The initial code for the generation of the consult question was as follows:

def gen_cq_gemini():
    model = genai.GenerativeModel('gemini-pro')
    question = """Fewshot Prompt Goes Here"""
    response = model.generate_content(question)
    return response.text

If you don't get a single Error: 500, this code works just fine. If an error does get thrown, using a try-except block will gracefully handle the error without crashing your program, and utilizing a while loop allows you to retry the API call before moving on.

Here's the code with a try-except block:

def gen_cq_gemini(max_retries = 4, delay = 2):
    retry_count = 0
    model = genai.GenerativeModel('gemini-pro')
    question = """Few-shot Prompt Goes Here"""
    while retry_count < max_retries:
        try:
           response = model.generate_content(question)
           return response.text
        except Exception as e:
           print(f"Error: {e}")
        time.sleep(delay)
        retry_count += 1
    return None

The try: block is the same as our initial code. Assuming the API returns a response without issue, the response is returned, and the function is terminated.

If an error is returned however, we are now able to handle it:

• The function continues into the except: block.
• The error is printed
• Pause for 2 seconds
• Increase the retry_count counter
• Return to the top of the loop to retry the API call
• If retry_count reaches max_retries, the function returns None and the program moves on

Using an incremental counter with a maximum number of retries prevents the loop from getting stuck.

For iterative natural language tasks, like generation of synthetic datasets, this solution allows the program to continue to function despite the occasional error.

Give Google Gemini's API a try, and if you find yourself hitting Error: 500 issues, throw a try-except loop in there to handle it.

If you found this helpful, subscribe to my newsletter, where I will be sharing the things I'm learning about Python, data science, machine learning and medical informatics. I am still a novice when it comes to data science and I welcome feedback; please let me know if any of my terminology is off or my explanations are misinformed.

Additional Resources:

• Google Gemini Python Quick Start