Lab2 · AI Agentic Engineering

#Welcome to the Second Lab - Week 1, Day 3

Today we will work with lots of models! This is a way to get comfortable with APIs.

Important point - please read

The way I collaborate with you may be different to other courses you've taken. I prefer not to type code while you watch. Rather, I execute Jupyter Labs, like this, and give you an intuition for what's going on. My suggestion is that you carefully execute this yourself, after watching the lecture. Add print statements to understand what's going on, and then come up with your own variations. See Q37 in the FAQ for how to set up a separate project for your work.

If you have time, I'd love it if you submit a PR for changes in the community_contributions folder - instructions in the resources. Also, if you have a Github account, use this to showcase your variations. Not only is this essential practice, but it demonstrates your skills to others, including perhaps future clients or employers...
And if you post about it on LinkedIn and tag me, then I'll weigh in to amplify your achievement. If you see other students posting, please give them your encouragement too.

In [1]:

# Start with imports - ask the Cursor Agent to explain any package that you don't know

import os
import json
from dotenv import load_dotenv
from openai import OpenAI
from IPython.display import Markdown, display

In [2]:

# Always remember to do this!
load_dotenv(override=True)

In [3]:

# Print the key prefixes to help with any debugging

openai_api_key = os.getenv('OPENAI_API_KEY')
anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')
google_api_key = os.getenv('GOOGLE_API_KEY')
deepseek_api_key = os.getenv('DEEPSEEK_API_KEY')
groq_api_key = os.getenv('GROQ_API_KEY')
grok_api_key = os.getenv('GROK_API_KEY')
openrouter_api_key = os.getenv('OPENROUTER_API_KEY')

if openai_api_key:
    print(f"OpenAI API Key exists and begins {openai_api_key[:8]}")
else:
    print("OpenAI API Key not set")
    
if anthropic_api_key:
    print(f"Anthropic API Key exists and begins {anthropic_api_key[:7]}")
else:
    print("Anthropic API Key not set (and this is optional)")

if google_api_key:
    print(f"Google API Key exists and begins {google_api_key[:2]}")
else:
    print("Google API Key not set (and this is optional)")

if deepseek_api_key:
    print(f"DeepSeek API Key exists and begins {deepseek_api_key[:3]}")
else:
    print("DeepSeek API Key not set (and this is optional)")

if groq_api_key:
    print(f"Groq API Key exists and begins {groq_api_key[:4]}")
else:
    print("Groq API Key not set (and this is optional)")

if grok_api_key:
    print(f"Grok API Key exists and begins {grok_api_key[:4]}")
else:
    print("Grok API Key not set (and this is optional)")

if openrouter_api_key:
    print(f"OpenRouter API Key exists and begins {openrouter_api_key[:6]}") 
else:
    print("OpenRouter API Key not set (and this is optional)")

In [4]:

request = """
Please come up with a challenging, nuanced question with a succinct answer,
that I can ask a number of LLMs to evaluate their intelligence.
Not a mathematical puzzle, but more of a thought-provoking question that requires intelligent insight.
Include in your question that the answer must be short.
"""
request += "Answer only with the question, no explanation."
messages = [{"role": "user", "content": request}]

In [5]:

messages

In [6]:

openai = OpenAI()

response = openai.chat.completions.create(model="gpt-5.4-mini", messages=messages)
question = response.choices[0].message.content
display(Markdown(question))

#Calling LLMs from multple providers

We are about to call LLMs from many other providers. They all provide API endpoints that are compatible with OpenAI, as explained in Guide 9 in the guides folder. So we can simply use these endpoints as if we are using OpenAI.

Please note:

I'm going to use lots of LLMs from different providers, but you don't need to! This is only to show their abilities.

In [7]:

# OpenAI Compatible URLs

ANTHROPIC_BASE_URL = "https://api.anthropic.com/v1/"
DEEPSEEK_BASE_URL = "https://api.deepseek.com/v1"
GEMINI_BASE_URL = "https://generativelanguage.googleapis.com/v1beta/openai/"
GROQ_BASE_URL = "https://api.groq.com/openai/v1"
GROK_BASE_URL = "https://api.x.ai/v1"
OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1"
OLLAMA_BASE_URL = "http://localhost:11434/v1"

In [8]:

# OpenAI client libraries with the right base_url and key
# If this surprises you, please see Guide 9 in the Guides folder!

anthropic = OpenAI(api_key=anthropic_api_key, base_url=ANTHROPIC_BASE_URL)
deepseek = OpenAI(api_key=deepseek_api_key, base_url=DEEPSEEK_BASE_URL)
gemini = OpenAI(api_key=google_api_key, base_url=GEMINI_BASE_URL)
groq = OpenAI(api_key=groq_api_key, base_url=GROQ_BASE_URL)
grok = OpenAI(api_key=grok_api_key, base_url=GROK_BASE_URL)
openrouter = OpenAI(api_key=openrouter_api_key, base_url=OPENROUTER_BASE_URL)
ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key="ollama")

In [9]:

competitors = []
answers = []
messages = [{"role": "user", "content": question}]

In [10]:

def record(model_name, answer):
    competitors.append(model_name)
    answers.append(answer)
    display(Markdown(answer))

In [11]:

# The API we know well
# Reasoning effort can be none, low, medium, high, or xhigh

model_name = "gpt-5.4-nano"

response = openai.chat.completions.create(model=model_name, messages=messages, reasoning_effort="none")
answer = response.choices[0].message.content

record(model_name, answer)

In [12]:

model_name = "claude-sonnet-4-6"

response = anthropic.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

record(model_name, answer)

In [13]:

model_name = "gemini-3.1-flash-lite"

response = gemini.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

record(model_name, answer)

In [14]:

model_name = "deepseek-v4-flash"

response = deepseek.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

record(model_name, answer)

In [15]:

model_name = "openai/gpt-oss-120b"

response = groq.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

display(Markdown(answer))
competitors.append(model_name)
answers.append(answer)

In [16]:

model_name = "moonshotai/kimi-k2.6"

response = openrouter.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

record(model_name, answer)

#For the next cell, we will use Ollama

Ollama runs a local web service that gives an OpenAI compatible endpoint,
and runs models locally using high performance C++ code.

If you don't have Ollama, install it here by visiting https://ollama.com then pressing Download and following the instructions.

After it's installed, you should be able to visit here: http://localhost:11434 and see the message "Ollama is running"

You might need to restart Cursor (and maybe reboot). Then open a Terminal (control+`) and run ollama serve

Useful Ollama commands (run these in the terminal, or with an exclamation mark in this notebook):

ollama pull <model_name> downloads a model locally
ollama ls lists all the models you've downloaded
ollama rm <model_name> deletes the specified model from your downloads

Super important - ignore me at your peril!

Many models on Ollama are FAR too large for your home computer. Be sure to browse the models on the Ollama website. Look to use models that are size 3GB or less unless you know better; llama3.2 is a great first choice. Don't pick models that end in :cloud; that's something different (a cloud inference service, like Groq).

In [17]:

!ollama pull llama3.2

In [18]:

import requests
requests.get('http://localhost:11434').content

In [19]:

import requests
models = requests.get('http://localhost:11434/v1/models').json()
for model in models.get("data"):
    print(model.get("id"))

In [20]:

model_name = "llama3.2:1b"

response = ollama.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

record(model_name, answer)

In [21]:

model_name = "gpt-oss:latest"

response = ollama.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

display(Markdown(answer))
competitors.append(model_name)
answers.append(answer)

In [22]:

model_name = "gemma4:latest"

response = ollama.chat.completions.create(model=model_name, messages=messages)
answer = response.choices[0].message.content

display(Markdown(answer))
competitors.append(model_name)
answers.append(answer)

In [23]:

# So where are we?

print(len(competitors))
print(competitors)
print(answers)

In [24]:

# It's nice to know how to use "zip"
for competitor, answer in zip(competitors, answers):
    print(f"Competitor: {competitor}\n\n{answer}")

In [25]:

# Let's bring this together - note the use of "enumerate"

together = ""
for index, answer in enumerate(answers):
    together += f"# Response from competitor {index+1}\n\n"
    together += answer + "\n\n"

In [26]:

print(together)

In [27]:

judge = f"""You are judging a competition between {len(competitors)} competitors.
Each model has been given this question:

{question}

Your job is to evaluate each response for clarity and strength of argument, and rank them in order of best to worst.
Respond with JSON, and only JSON, with the following format:
{{"results": ["best competitor number", "second best competitor number", "third best competitor number", ...]}}

Here are the responses from each competitor:

{together}

Now respond with the JSON with the ranked order of the competitors, nothing else. Do not include markdown formatting or code blocks."""

In [28]:

print(judge)

In [29]:

judge_messages = [{"role": "user", "content": judge}]

#And now for Grok!

Branded as "The most truth-seeking large language model in the world".. so let's use it as our LLM as a judge

In [30]:

# Judgement time!
# Grok is "The most truth-seeking large language model in the world."

model_name = "grok-4.3"

response = grok.chat.completions.create(model=model_name, messages=judge_messages)
results = response.choices[0].message.content
print(results)

In [31]:

# OK let's turn this into results!

results_dict = json.loads(results)
ranks = results_dict["results"]
for index, result in enumerate(ranks):
    competitor = competitors[int(result)-1]
    print(f"Rank {index+1}: {competitor}")

Exercise

Which pattern(s) did this use? Try updating this to add another Agentic design pattern.

Commercial implications

These kinds of patterns - to send a task to multiple models, and evaluate results, are common where you need to improve the quality of your LLM response. This approach can be universally applied to business projects where accuracy is critical.