meta/llama-2-70b-chat

A 70 billion parameter language model from Meta, fine tuned for chat completions

Capabilities

SeedSystem PromptMax TokensTop-P

Cost

Community model (estimated from hardware time)

Input Parameters

Name	Type	Description	Default	Constraints
`prompt`*	string	Prompt to send to the model.	`—`	—
`debug`	boolean	provide debugging output in logs	`false`	—
`max_new_tokens`	integer	Maximum number of tokens to generate. A word is generally 2-3 tokens	`128`	min: 1
`min_new_tokens`	integer	Minimum number of tokens to generate. To disable, set to -1. A word is generally 2-3 tokens.	`-1`	min: -1
`replicate_weights`	string	Path to fine-tuned weights produced by a Replicate fine-tune job.	`—`	—
`seed`	integer	Random seed. Leave blank to randomize the seed	`—`	—
`stop_sequences`	string	A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.	`—`	—
`system_prompt`	string	System prompt to send to the model. This is prepended to the prompt and helps guide system behavior.	`"You are a helpful assistant."`	—
`temperature`	number	Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.	`0.75`	min: 0.01, max: 5
`top_k`	integer	When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens	`50`	min: 0
`top_p`	number	When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens	`0.9`	min: 0, max: 1

promptrequiredstring

Prompt to send to the model.

debugboolean

provide debugging output in logs

Default: false

max_new_tokensinteger

Maximum number of tokens to generate. A word is generally 2-3 tokens

Default: 128min: 1

min_new_tokensinteger

Minimum number of tokens to generate. To disable, set to -1. A word is generally 2-3 tokens.

Default: -1min: -1

replicate_weightsstring

Path to fine-tuned weights produced by a Replicate fine-tune job.

seedinteger

Random seed. Leave blank to randomize the seed

stop_sequencesstring

A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of 'end' or '<stop>'.

system_promptstring

System prompt to send to the model. This is prepended to the prompt and helps guide system behavior.

Default: "You are a helpful assistant."

temperaturenumber

Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value.

Default: 0.75min: 0.01, max: 5

top_kinteger

When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens

Default: 50min: 0

top_pnumber

When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens

Default: 0.9min: 0, max: 1

Version: 02e509c78996Updated: 7/25/202610.1M runs