AI2025-02-0310 min read

White Paper: Amazon Bedrock for building AI conversations - The cost of memory, tokens, and hosting

The costings presented here are in Dollars as this is what is highlighted on the AWS pricing guidelines. Amazon Bedrock offers access to Meta's Llama 3.2 models, which...

The costings presented here are in Dollars as this is what is highlighted on the AWS pricing guidelines.

Amazon Bedrock offers access to Meta's Llama 3.2 models, which are available in various configurations to suit different application needs. Here's an overview of the available models and their pricing:

Llama 3.2 Model Variants and Pricing:

Model | Parameters | Pricing per 1,000 Tokens |

Llama 3.2 Instruct | 1B | $0.0001 |

Llama 3.2 Instruct | 3B | $0.0002 |

Llama 3.2 Instruct | 11B | $0.00035 |

Llama 3.2 Instruct | 90B | $0.002 |

Note: Pricing includes both input and output tokens. nops.io Token Usage and Context Management:

Llama 3.2 models support a context window of up to 128,000 tokens, allowing them to handle extensive conversations and large inputs without the need to repeatedly include previous messages in each request. This capability enables the model to maintain context over long interactions, reducing token consumption and associated costs. aws.amazon.com Incorporating Retrieval-Augmented Generation (RAG):

Integrating Retrieval-Augmented Generation involves combining the Llama 3.2 model with a retrieval system to access external data sources, enhancing the model's responses with up-to-date information. While Amazon Bedrock facilitates such integrations, the specific costs depend on the retrieval system employed and the volume of data processed. It's important to consider both the computational resources required for the retrieval process and the inference costs of the Llama 3.2 model.

Cost Estimation for Simulating an Interview Process:

To estimate the cost of using Llama 3.2 for an interview simulation that includes a custom prompt, an ongoing conversation, and RAG integration, consider the following:

Custom Prompt and Conversation:

Token Count: Estimate the total number of tokens for the initial prompt and the expected dialogue. For instance, a 500-token prompt followed by a 1,500-token conversation totals 2,000 tokens.

Model Selection: Choose an appropriate model based on complexity. For detailed simulations, the Llama 3.2 Instruct 11B model might be suitable.
Cost Calculation: At $0.00035 per 1,000 tokens, processing 2,000 tokens would cost approximately $0.0007 per session.
RAG Integration:

Retrieval Costs: These depend on the external system used. If utilizing AWS services like Amazon OpenSearch or DynamoDB, factor in their respective costs based on usage.

Additional Token Processing: Incorporating retrieved data may increase the token count, slightly raising the inference cost.

Total Estimated Cost per Interview Session:

Inference Cost: Approximately $0.0007 per session for the Llama 3.2 Instruct 11B model.
Retrieval System Costs: Variable, depending on the chosen system and data volume.

For a precise cost assessment, it's advisable to monitor actual token usage during a test run and consult the pricing details of any additional AWS services employed in the RAG setup.

Understanding Token Usage in Conversations on Amazon Bedrock with Llama 3.2

In Amazon Bedrock, when you interact with Llama 3.2, the entire conversation history must be included in each request because the model itself does not maintain session memory between API calls. However, since Llama 3.2 supports a large context window (up to 128,000 tokens), it can process and "remember" previous messages within that window.

1. How Conversation Context is Stored and Processed

Each request you send to Llama 3.2 includes all previous messages within the allowed context window.
Amazon Bedrock does not provide free context memory—you are charged for every token that is sent in a request, including the tokens from previous messages that you include to maintain context.
The context window (e.g., 128,000 tokens) allows you to fit a large portion of an ongoing conversation before older messages need to be removed.

2. Cost Breakdown Over a Conversation

Assume the following scenario for a 1500-token conversation (excluding the initial prompt):

Scenario: Ongoing Interview Simulation

Prompt (500 tokens)
Each user input (100 tokens)
Each model response (200 tokens)
Conversation lasts 10 turns (10 user inputs & 10 model responses)

1 | 500 + 100 | 200 | 800 | 800 |

2 | 500 + 100 + 200 + 100 | 200 | 1100 | 1900 |

3 | 500 + 100 + 200 + 100 + 200 + 100 | 200 | 1400 | 3300 |

4 | 500 + 100 + 200 + 100 + 200 + 100 + 200 + 100 | 200 | 1700 | 5000 |

5 | 500 + 100 + 200 + 100 + 200 + 100 + 200 + 100 + 200 + 100 | 200 | 2000 | 7000 |

Key Observations:

With each new response, the total number of tokens included in the request grows, because it must contain all past conversation turns.
This means costs increase exponentially over time until the context window reaches 128,000 tokens.
If the context window is exceeded, older messages must be dropped, meaning some memory of the conversation will be lost.

3. Cost Calculation Over a 10-Turn Conversation

Now, let's estimate the costs using Llama 3.2 Instruct 11B at $0.00035 per 1,000 tokens.

Total tokens used (from table above, including user inputs and responses): ~ 7000 tokens
Cost per request:

$0.00035 × (7000 / 1000) = $0.00245 per session

Cost per 100 interview sessions:

$0.00245 × 100 = $0.245 (~25 cents)

If the interview is longer (e.g., 20+ turns), costs increase significantly due to the growing token count.

4. Does AWS Provide the Context Window for Free?

No, Amazon Bedrock does not provide free session memory.

Unlike OpenAI’s chat-based APIs, which maintain memory per conversation thread, AWS requires you to pass past messages manually within each API call.
Every token in the request is billable (both input and output tokens).

5. How Much Would a RAG Integration Cost?

Adding Retrieval-Augmented Generation (RAG) means:

You must retrieve external documents and inject them into each API request.
If the retrieved document is 2,000 tokens, this significantly increases token usage per call.

For a RAG-based interview:

Each turn may include:

Context from previous conversation (increasing token count)

Retrieved documents (~2000 extra tokens per request)
User input + model response (~300 tokens)

Cost Estimate for RAG-enhanced Conversation:

1 | 800 | 2000 | 2800 | $0.00098 |

2 | 1100 | 2000 | 3100 | $0.00109 |

3 | 1400 | 2000 | 3400 | $0.00119 |

4 | 1700 | 2000 | 3700 | $0.00130 |

For a 10-turn RAG conversation:

Estimated cost: $0.015 per session
100 sessions: $1.50

6. Final Cost Estimate for a Full Interview Process

Feature | Cost Per Session |

Llama 3.2 (Non-RAG, 10-turns) | $0.0025 |

Llama 3.2 + RAG (10-turns) | $0.015 |

100 Non-RAG Interviews | $0.25 |

100 RAG-Based Interviews | $1.50 |

If you scale to 1,000 interviews, a RAG-enhanced process would cost ~$15, while a basic conversation would cost ~$2.50.

7. Optimizing Costs

To reduce costs:

Use a smaller model (e.g., Llama 3.2 3B model instead of 11B).
Summarize older messages instead of passing the full history.
Limit the number of retrieved documents in RAG.

Final Thoughts

Amazon Bedrock requires you to pass the full conversation context manually, increasing token costs per turn.
A longer conversation means higher costs, as each API request contains more tokens.
RAG adds extra tokens (~2000 per query), further increasing costs.
For a 10-turn conversation, expect ~$0.0025 per session for standard Llama 3.2 (11B) and ~$0.015 per session with RAG.

Here’s a detailed breakdown of different use cases where Llama 3.2 on Amazon Bedrock could be used, including cost estimations for each scenario.

1. Standard Interview Process (Single User)

Use Case: A simulated interview conversation between an AI interviewer and a single user.

Token Usage Assumptions:

Prompt: 500 tokens
User Input Per Turn: 100 tokens
AI Response Per Turn: 200 tokens
Total Turns: 10

1 | 500 + 100 | 200 | 800 | 800 |

2 | 800 + 100 | 200 | 1100 | 1900 |

3 | 1100 + 100 | 200 | 1400 | 3300 |

... | ... | ... | ... | ... |

10 | ~4400 + 100 | 200 | ~4700 | ~25000 |

Estimated Cost for Llama 3.2 (11B):

Total tokens: ~25,000 per interview
Cost per 1,000 tokens: $0.00035
Cost per interview: $0.00875
Cost for 1,000 interviews: $8.75

2. RAG-Enhanced Interview

Use Case: AI interviews users with access to external documents (e.g., company policies, job descriptions) using Retrieval-Augmented Generation (RAG).

Token Usage Assumptions:

Base Tokens Per Turn (User Input + AI Response + Context): 300 tokens
RAG Retrieved Document Per Turn: ~2,000 tokens
Total Turns: 10

1 | 800 | 2000 | 2800 | 2800 |

2 | 1100 | 2000 | 3100 | 5900 |

3 | 1400 | 2000 | 3400 | 9300 |

10 | ~4400 | 2000 | ~6400 | ~55,000 |

Estimated Cost for Llama 3.2 (11B) with RAG:

Total tokens: ~55,000 per interview
Cost per 1,000 tokens: $0.00035
Cost per interview: $0.01925
Cost for 1,000 interviews: $19.25
Additional costs for RAG system (e.g., OpenSearch, DynamoDB): Depends on retrieval volume, storage.

3. Multi-User Panel Interview (3 Participants + AI)

Use Case: AI moderates a panel interview with multiple participants.

Token Usage Assumptions:

Prompt: 500 tokens
Each User Input (3 users per turn): 100 tokens × 3 = 300 tokens
AI Response: 400 tokens
Total Turns: 10

1 | 500 + 300 | 400 | 1200 | 1200 |

2 | 1200 + 300 | 400 | 1900 | 3100 |

3 | 1900 + 300 | 400 | 2600 | 5700 |

10 | ~5000 + 300 | 400 | ~5700 | ~34,000 |

Estimated Cost for Llama 3.2 (11B):

Total tokens: ~34,000 per interview
Cost per 1,000 tokens: $0.00035
Cost per interview: $0.0119
Cost for 1,000 interviews: $11.90

If RAG is added, expect an extra 20,000 tokens per session, pushing the cost per 1,000 interviews to ~$20.

4. AI-Driven Coding Interview with Code Execution

Use Case: AI asks coding questions, evaluates responses, and runs code snippets.

Token Usage Assumptions:

Prompt: 500 tokens
User Input (Code + Explanation): ~300 tokens
AI Evaluation Response: 400 tokens
Code Execution Logs (RAG from an external execution system): 3,000 tokens per run
Total Turns: 5 (with 2 code execution requests)

1 | 500 + 300 | 400 | 0 | 1200 |

2 | 1200 + 300 | 400 | 0 | 1900 |

3 | 1900 + 300 | 400 | 3000 | 5600 |

5 | ~5600 + 300 | 400 | 3000 | ~10,000 |

Estimated Cost for Llama 3.2 (11B):

Total tokens: ~10,000 per session
Cost per 1,000 tokens: $0.00035
Cost per interview: $0.0035
Cost for 1,000 interviews: $3.50
Code Execution Costs: Additional AWS Lambda or EC2 charges.

5. Real-Time Customer Support Chatbot

Use Case: AI provides context-aware support to users over multiple interactions.

Token Usage Assumptions:

Prompt: 500 tokens
User Input Per Turn: 50 tokens
AI Response Per Turn: 100 tokens
Total Turns: 20
RAG (Product Manuals, FAQs, etc.): 2,000 tokens per retrieval (every 5 turns)

1 | 500 + 50 | 0 | 650 | 650 |

5 | 2200 + 50 | 2000 | 4250 | 10,600 |

10 | ~5000 + 50 | 2000 | 7050 | 21,200 |

20 | ~9500 + 50 | 2000 | 11,050 | 41,300 |

Estimated Cost for Llama 3.2 (11B) with RAG:

Total tokens: ~41,300 per session
Cost per 1,000 tokens: $0.00035
Cost per session: $0.0145
Cost for 1,000 sessions: $14.50
Additional costs: Storage and retrieval via Amazon Kendra, OpenSearch, or DynamoDB.

6. AI-Powered Debate System

Use Case: AI argues with itself or facilitates a debate between two users.

Token Usage Assumptions:

AI Opening Statement: 500 tokens
User Arguments (Per Turn): 150 tokens
AI Rebuttals (Per Turn): 200 tokens
Total Turns: 10 (5 rounds)

1 | 500 + 150 | 200 | 850 | 850 |

5 | ~2800 + 150 | 200 | ~3150 | ~11,000 |

Estimated Cost for Llama 3.2 (11B):

Total tokens: ~11,000 per session
Cost per interview: $0.00385
Cost for 1,000 sessions: $3.85

Conclusion

Basic interview: $8.75 per 1,000
RAG interview: $19.25 per 1,000
Multi-user panel: $11.90 per 1,000
Coding interview: $3.50 per 1,000
Support chatbot: $14.50 per 1,000
Debate system: $3.85 per 1,000

Save Time in Schools & Colleges

AI avatars deliver interview skills at scale.

Back to resources