AI on Demandpowered by BAAI

bge-m3 and bge-reranker-v2-m3 from BAAI: two specialised models for embedding and reranking, supporting over 100 languages and up to 8,192 tokens. stepping stone runs both on Swiss infrastructure — as the foundation for search and RAG architectures that are not dependent on the US.

stepping stone provides two models from the Beijing Academy of Artificial Intelligence (BAAI): bge-m3 for embedding and bge-reranker-v2-m3 for reranking.

The embedding model converts text into vectors, enabling powerful, cross-lingual search — across over 100 languages, from short queries to documents containing up to 8,192 tokens. The reranking model sorts results by relevance and improves the quality of search results.

Combined with one of stepping stone’s LLM offerings, this creates a Retrieval-Augmented Generation (RAG) pipeline: your internal data becomes available within the context of every query — accurately, efficiently and without leaving the country.

Development teams and companies looking to build intelligent search capabilities using internal data or to enhance existing AI applications with their own knowledge. Particularly suitable for organisations with multilingual document collections or compliance requirements.

Typical use cases: semantic search across knowledge databases and archives, RAG pipelines for chatbots and assistant systems, multilingual document search, and improving the quality of existing search solutions through reranking.

Open source (MIT / Apache 2.0). Swiss data centres. No data stored with US providers.

Embedding and reranking require significantly less computing power than an LLM that would have to perform the same task — this saves on tokens and costs. Modern API with comprehensive documentation and clear examples. Personalised advice and operation provided by stepping stone in Bern.

Your Consultant

Scope of services

Embedding and reranking on demand

Access to bge-m3 for multilingual vector search and bge-reranker-v2-m3 for precise re-ranking. Over 100 languages, up to 8,192 tokens per document.

RAG-compatible infrastructure

Can be combined with stepping stone’s LLM offerings to create a complete RAG pipeline. Your internal data will be available in the context of each query, without ever leaving the country.

Managed service

Deployment, monitoring, maintenance and support on Swiss infrastructure, with personalised advice. stepping stone takes care of the day-to-day running so that you can focus on the benefits.

Areas of application

Semantic search

BAAI models enable a search that understands meaning — rather than simply matching keywords.

Companies use bge-m3 to make knowledge bases, document archives and internal repositories semantically searchable. The re-ranking model then improves the quality of search results through precise relevance weighting — multilingual, efficient and hosted on Swiss infrastructure.

RAG Pipelines

Embedding and reranking are the core components of any retrieval-augmented generation pipeline.

When combined with one of stepping stone’s LLM solutions, RAG systems are created that integrate internal data into every query in a contextually accurate manner. Compatible with LangChain and LlamaIndex — without leaving the country.

Benchmark

bge-m3

The benchmarks were measured using the vllm bench tool against the production API gateway. The standard input sizes were 512 tokens. This equates to around 350-400 words.

 

Call

# Set your personal key:
STONEY_KEY=sk-...

# Make key visible for vllm bench:
export OPENAI_API_KEY=$STONEY_KEY

# Start the benchmark
vllm bench serve \
  --backend openai-embeddings \
  --model "BAAI/bge-m3" \
  --base-url llm.stoney-cloud.com \
  --endpoint /v1/embeddings \
  --dataset-name random \
  --tokenizer "BAAI/bge-m3" \
  --random-input-len 512 \
  --random-batch-size 1 \
  --num-prompts 100 \
  --max-concurrency 1

 

Result

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Maximum request concurrency:             1
Benchmark duration (s):                  1.29
Total input tokens:                      51200
Request throughput (req/s):              77.61
Total token throughput (tok/s):          39734.80
----------------End-to-end Latency----------------
Mean E2EL (ms):                          12.39
Median E2EL (ms):                        12.48
P99 E2EL (ms):                           14.66
==================================================

bge-reranker-v2-m3

The benchmarks were measured using the vllm bench tool against the production API gateway. For each query, 512 tokens were used as input, with a batch size of 5 documents per search query.

 

Call

# Set your personal key:
STONEY_KEY=sk-...

# Make key visible for vllm bench:
export OPENAI_API_KEY=$STONEY_KEY

# Start the benchmark
vllm bench serve \
  --backend vllm-rerank \
  --model BAAI/bge-reranker-v2-m3 \
  --base-url llm.stoney-cloud.com \
  --endpoint /v1/rerank \
  --dataset-name random-rerank \
  --tokenizer BAAI/bge-reranker-v2-m3 \
  --random-input-len 512 \
  --num-prompts 500 \
  --random-batch-size 1

Result

============ Serving Benchmark Result ============
Successful requests:                     100
Failed requests:                         0
Benchmark duration (s):                  1.25
Total input tokens:                      256500
Request throughput (req/s):              80.13
Total token throughput (tok/s):          205531.35
----------------End-to-end Latency----------------
Mean E2EL (ms):                          1025.72
Median E2EL (ms):                        1214.52
P99 E2EL (ms):                           1235.71
==================================================

 

Legend

  • Successful requests: Successful prompt requests.
  • Failed requests: Unsuccessful prompts.
  • Maximum request concurrency: How many requests the model processes simultaneously.
  • Benchmark duration (s): The duration of the benchmark run in seconds.
  • Total input tokens: The total number of input tokens.
  • Request throughput (req/s): The number of requests processed per second.
  • Total token throughput (tok/s): The average of all tokens processed during the measurement.
  • Mean End to End Latency (E2EL) (ms): The average time between input and completed output.
  • Median E2EL (ms): The typical time between input and completed output.
  • p99 E2EL (ms): The "worst case" time until completed output.
  • Batch-size: For embeddings, the number of texts embedded together in a single request.

Price

ModelContext lengthMTok
bge-m3                    8k0.1000

ModelContext lengthMTok
bge-reranker-v2-m38k0.1000
All prices are in CHF/MTok, excluding VAT.