Super Fast Responses with Groq Cloud

Try the Groq Cloud beta to get near instantaneous responses from Steve

1 min read
Cover Image for Super Fast Responses with Groq Cloud

Groq is a company that makes chips customized to run LLMs, and they run very fast.

They have a cloud offering that is currently in free beta. Connect to it and see just how fast Steve can be!

  1. Create a Groq Cloud account
  2. Create an API key
  3. Go to the Models page in Ask Steve Settings, press ADD NEW MODEL and select OpenAI GPT-3.5 from the menu. Groq Cloud's API is modeled on OpenAI's, so we'll use it as a starting point. Go ahead and paste in your Groq Cloud API Key.
  4. Change the Name to Groq Cloud, the URL to https://api.groq.com/openai/v1/chat/completions and in the Body change model to llama3-70b-8192. You can see all the available models here and their rate limits here
  5. Press TEST to ensure it works, then SAVE NEW MODEL to save it.
  6. Steve is now SUPER FAST 馃殌馃殌馃殌

Read more about

Cover Image for How Steve Got His Name
1 min read馃摪News

Infinite Steves, at your service