Can your numbers be recalculated?

Yes. The runs are published question by question, with a verify.js script. It redoes the benchmark's exact calculation: average per conversation, then per category, then across the ten, and a Kendall tau where the benchmark asks for one. A simple average would give a different number. We broke the data three times to check that the script catches the error.

What exactly is being audited?

The grading. The questions and reference answers belong to LongMemEval and to BEAM, both under the MIT licence. The engine that produced the answers is closed.

What machine does it run on, and where does our corpus go?

Yours. Mnemosyne OS is a desktop application. Search and vectorization run locally, and a GPU speeds them up if you have one. Your folders are read in place and never modified. With a local model, your documents and your questions stay on the machine. With a cloud model, the question and the relevant excerpts go to that provider, under your key or your Vertex project.

Open core. The SDK, the MCP server and the npm packages are under the MIT licence, readable by anyone. The memory engine is closed: that is what we sell. Command-line access to the core is discussed as part of a partnership.

Our lab is not a memory lab. Does this concern us?

Yes, that is exactly the offer. We are looking for labs in medicine, biology, chemistry, or any discipline with a large corpus, to build a dedicated engine with them. You bring the expertise and the corpus, we bring the engine. Everything starts with a conversation about how you work.

Dossier 02 · Research labs

A measured memory engine for research labs

Our memory engine is measured on two public benchmarks, with the ledgers published. Every question has its verdict. Our failures count as much as our scores.

Do you work on long-term memory, information retrieval or human-AI interaction? Do you want to evaluate, replicate or collaborate? Write to us.

Download Mnemosyne OS See the application

Windows, macOS and Linux.

Protect your research

100% local for unpublished work, hybrid for the rest

100% local

Everything on the machine

The model runs inside the application (Qwen, Gemma, Phi in GGUF). Unplug the network cable: search and answers keep working.

Hybrid

Search local, answer in the cloud

Search runs on the machine. A cloud model writes the answer, with your own key or your Vertex project. The question and the retrieved passages go to that provider.

Your server

Your lab’s model

Your IT already runs a model? Connect it: any server that speaks the OpenAI-compatible API, at an address you set in the settings. The question and the passages go to that machine.

Compatible models

Vertex AI
Gemini
OpenAI
Anthropic
Mistral
DeepSeek
MiniMax
Qwen (local)
Your server

A lab can work fully offline on a sensitive corpus, and in hybrid mode on public papers. Download Mnemosyne OS to try it on your corpus. For a deployment on the lab’s server, talk to us.

Download Mnemosyne OS → Talk to us →

Working as a research team

Share a corpus across the lab, with no server Beta

Every computer in the lab runs Mnemosyne OS. On the same network, they connect directly, with no server in the middle.

You lend a colleague a vault to read, or you share a corpus with the team: each person writes to it and keeps their own copy. The computers can also send each other messages.

A deletion never spreads. If two versions of a file diverge, both are kept. Between two sites, the application sets up Tailscale and guides you.

The network stays off until someone turns it on, computer by computer.

Fig. · Four computers in the lab, one shared corpus

What is measured

LongMemEval-M

77.1%

48 questions it had never seen, full-haystack, official judge. Every answer published, the misses included.
BEAM · 100K

61.7%

400 questions across ten capabilities, graded by the benchmark's official judge.
BEAM · 10M

49.2%

200 questions on a corpus 76 times larger. Same judge, same pipeline, no tuning in between.
The slope

20% lost

For a corpus 76 times larger. In the BEAM paper, the RAG baselines lose 22 to 29% on the same jump. The pair of figures matters more than a single score.

These numbers read next to the category’s →

Collaboration offer

A memory engine for your discipline: medicine, biology, chemistry

The offer

We build, with labs, a memory engine for their field: medicine, biology, chemistry, any discipline with a large corpus. You bring the expertise and the corpus, we bring the engine. It is a collaboration, not a ready-made product.

What we want to build

A tool that watches new publications on your subjects, on arXiv and elsewhere, and adds them to your memory. At night it looks for links between your corpus and your peers’ work, on your machine.

How it starts

With a conversation: how you work, what slows you down, what takes up your weeks. The first researchers who test it decide the shape of the tool.

Our method

Protocols pre-registered before the decisive tests, control arms, permutation tests with a fixed seed. The founder’s favourite hypothesis was refuted, and the refutation is published.

Internal research program: blind pre-registration · control arms · falsification is part of the method

Questions

What a lab asks us first

Can your numbers be recalculated?: Yes. The runs are published question by question, with a verify.js script. It redoes the benchmark's exact calculation: average per conversation, then per category, then across the ten, and a Kendall tau where the benchmark asks for one. A simple average would give a different number. We broke the data three times to check that the script catches the error.
What exactly is being audited?: The grading. The questions and reference answers belong to LongMemEval and to BEAM, both under the MIT licence. The engine that produced the answers is closed.
What machine does it run on, and where does our corpus go?: Yours. Mnemosyne OS is a desktop application. Search and vectorization run locally, and a GPU speeds them up if you have one. Your folders are read in place and never modified. With a local model, your documents and your questions stay on the machine. With a cloud model, the question and the relevant excerpts go to that provider, under your key or your Vertex project.
Is it open source?: Open core. The SDK, the MCP server and the npm packages are under the MIT licence, readable by anyone. The memory engine is closed: that is what we sell. Command-line access to the core is discussed as part of a partnership.
Our lab is not a memory lab. Does this concern us?: Yes, that is exactly the offer. We are looking for labs in medicine, biology, chemistry, or any discipline with a large corpus, to build a dedicated engine with them. You bring the expertise and the corpus, we bring the engine. Everything starts with a conversation about how you work.

Cite this work Technical whitepaper, DOI 10.5281/zenodo.21728283 Benchmark audit kit, DOI 10.5281/zenodo.21727140 ORCID 0009-0009-1087-3917 BEAM ledgers, both runs question by question

← Back to the audience router