开源和本地 LLM 的结构化输出¶

最初，Instructor 仅通过 OpenAI SDK 促进 API 交互，并通过集成 Zod 进行结构化数据验证和序列化，重点关注函数调用。

随着时间的推移，我们通过集成 JSON 模式扩展了我们的工具包，从而增强了我们对视觉模型和开源模型的适应性。这一进步使我们现在能够支持广泛的模型，从 GPT 和 Mistral 到几乎所有可以通过 Ollama 和 Hugging Face 访问的模型。有关如何利用 JSON 模式与各种模型的更多见解，请参阅我们关于 Patching 的详细指南。

探索不同 OpenAI 客户端与 Instructor 的集成¶

下面，我们探索了一些与 Instructor 集成的值得注意的客户端，它们提供了结构化输出和增强的功能，并附有如何初始化和修补每个客户端的示例。

本地模型¶

Ollama：本地模型的新领域¶

要深入了解 Ollama，包括设置和高级功能，请参阅文档。Ollama 官方网站还为新手提供了基本资源、模型下载和社区支持。

ollama run llama2

import Instructor from "@instructor-ai/instructor"
import OpenAI from "openai"
import { z } from "zod"

const UserExtractSchema = z.object({
  age: z.number(),
  name: z.string()
})

const oai = new OpenAI({
  apiKey: "ollama",  // required, but unused
  baseUrl: "https://:11434/v1", // updated API URL
})

const client = Instructor({
  client: oai,
  mode: "FUNCTIONS"
})

const user = await client.chat.completions.create({
  model: "llama2",
  messages: [{ role: "user", content: "Jason is 30 years old" }],
  response_model: { schema: UserExtractSchema, name: "UserExtractSchema" }
})

console.log(user)
// { age: 30, name: "Jason" }

其他提供商¶

Anyscale¶

export ANYSCALE_API_KEY="your-api-key"

import { z } from "zod";
import Instructor from "@instructor-js/instructor";
import OpenAI from "openai";

// Define the schema using Zod
const UserExtractSchema = z.object({
  name: z.string(),
  age: z.number(),
});

// Initialize OpenAI client
const oai = new OpenAI({
  apiKey: process.env.ANYSCALE_API_KEY,
  base_url: "https://api.endpoints.anyscale.com/v1",
});

// Patch the OpenAI client with Instructor-js
const client = Instructor({
  client: oai,
  mode: "JSON_SCHEMA"
});

// Use the patched client to create a chat completion
const resp = await client.chat.completions.create({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  messages: [
    { role: "system", content: "You are a world class extractor" },
    { role: "user", content: 'Extract the following entities: "Jason is 20"' },
  ],
  response_model: { schema: UserExtractSchema, name: "UserExtractSchema" },
});

console.log(resp);
// Expected output: { name: 'Jason', age: 20 }

Groq¶

Groq 官方文档，以其张量架构提供了独特的处理方法。这项创新显著提高了结构化输出处理的性能。

export GROQ_API_KEY="your-api-key"

import { z } from "zod";
import Instructor from "@instructor-js/instructor";
import Groq from "groq-sdk";

// Initialize Groq client
const groqClient = new Groq({
  apiKey: process.env.GROQ_API_KEY,
});

// Define the schema using Zod
const UserExtractSchema = z.object({
  name: z.string(),
  age: z.number(),
});

// Patch the Groq client with Instructor-js
const client = Instructor({
  client: groqClient,
  mode: "FUNCTIONS",
});

// Use the patched client to create a chat completion
const user = await client.chat.completions.create({
  model: "mixtral-8x7b-32768",
  response_model: { schema: UserExtractSchema, name: "UserExtract" },
  messages: [
    { role: "user", content: "Extract jason is 25 years old" },
  ],
});

console.log(user);
// { name: 'jason', age: 25 }

Together AI¶

Together AI 与 Instructor 结合使用时，为希望在其应用中利用结构化输出的开发者提供了无缝体验。

export TOGETHER_API_KEY="your-api-key"

import { z } from "zod";
import Instructor from "@instructor-js/instructor";
import OpenAI from "openai";


const client = Instructor({
  client: new OpenAI({
    apiKey: process.env.TOGETHER_API_KEY,
    base_url: "https://api.together.xyz/v1",
  }),
  mode: "TOOLS",
});

const UserExtractSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const user = await client.chat.completions.create({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  response_model: { schema: UserExtractSchema, name: "UserExtract" },
  messages: [
    { role: "user", content: "Extract jason is 25 years old" },
  ],
});

console.assert(user instanceof UserExtractSchema, "Should be instance of UserExtract");
console.log(user);
//> name='jason', age=25