Step 1 : Sign Up and Log In
Visit the Confident AI DeepEval Console using the following URL:
🔗 https://www.confident-ai.com/
If you're new to the platform, create an account and log in to access the console.
Step 2 : Review the Documentation
For detailed setup and installation instructions, refer to the official documentation:
📘 https://www.confident-ai.com/docs/setup-and-installation
Step 3 : Run the Evaluation Code
Open a Google Colab notebook or Visual Studio Code (VSCode), and execute the provided code step by step.
In this evaluation, we’ll compare DeepSeek LLM with OpenAI’s Open-GPT-OSS LLM using the DeepEval framework.
Use Case
The test case focuses on evaluating a fraud detection mechanism in a banking system.
NOTE : Since the Ollama models are running locally on your machine, you need to use ngrok for port forwarding. This allows your Google Colab notebook to access the local server by exposing it through a public URL. But if you are running all this in VSCode locally then ngrok/port forwarding is not needed.
We begin by installing the essential Python packages for LangChain and model evaluation.