Mock Server for Local Development¶

The mock server is a local test server that mimics the DeepOrigin Platform API. It's useful for local development and testing without making real API calls to the platform.

Overview¶

The mock server (tests/mock_server/) provides mock responses for all API endpoints used by the DeepOriginClient. It runs locally using FastAPI and uvicorn, and serves responses based on fixture data stored in tests/fixtures/.

Architecture¶

The mock server is organized into routers, each handling a group of related endpoints:

All routers share in-memory stores (dicts/lists) that are created in MockServer.__init__ and passed into the router factory functions. This lets data flow between routers — for example, a tool execution in the tools router can inject records that are later visible via the data-platform router's result-explorer search.

Proteins (local only): Protein.sync() is wired through mock import-dataset register_protein to a single canonical row (MOCK_CANONICAL_PROTEIN_ID, tests/brd.pdb fixture) so IDs stay stable under --env local. There is no separate test module for the mock server; that behavior is exercised indirectly by any local test that syncs a protein (e.g. the registered_protein fixture).

SSE stream (local only): GET /sse/{org}/stream/{project} sends a retry: line, one filters.dirty frame naming every filter=<id>=<pattern> query param, then closes. Executions.wait_for_ingestion reads the execution row on connect, re-reads on that frame, and reconnects only if the row is still not terminal, so a row that reads DataIngesting once (see executions/search) resolves on the second read. While there is no row, the wait also reads the tools-service run and ends if the run failed. The stream for project MOCK_SSE_DISABLED_PROJECT_ID (tests/mock_server/constants.py) is refused with 404, as on a gateway without SSE, to test that the wait warns and polls the row instead.

Protein import (local only): an import-dataset csv_path run whose mapper targets proteins.*, as projects.sync_proteins sends, is ingested at submit (_ingest_import_dataset_protein_csv in routers/tools.py). Rows are read through the mapper's proteins.<field> keys, with header keys and cells stripped as the import tool does. A CSV with no rows ends the run Failed with a statusReason.

Protein zip (local only): with protein_zip_path, a row's file_path names a file in the uploaded zip, mirroring import-dataset 4.x (_protein_zip_entries and _register_zip_protein in routers/tools.py):

• Zip entries are matched by file name at any depth; the first match wins.
• Registrations are cached by that file name, so rows naming the same file share one protein. This is why build_protein_bundle gives every entry a unique internal name.
• Each file is stored at entities/proteins/<sha256><ext>. A project protein with the same content is reused and keeps its name. The mock hashes file bytes where the tool hashes coordinates.
• A missing file, or one without ATOM/HETATM lines, loses its file_path silently. The row is still imported as a protein with only its name, and the run ends Completed, as the tool does on dev.

Covered by tests/test_protein_folder_import_local.py.

Pose registration (local only): PoseSet.sync() runs a synchronous import-dataset process_sdf with register_poses, waits on client.executions.wait_for_ingestion, then reads the run's pose rows from result-explorer once. The mock covers each step:

• The import mirrors its jobOutputs["poses"] into result-explorer, each row with an id, the run's compute_job_id and the SDF record_index. The run's data-platform row reads DataIngesting on the first executions/search and Completed after that.
• An SDF record named MOCK_POSE_REGISTRATION_FAIL_NAME (routers/tools.py) makes the run end Failed with statusReason MOCK_POSE_REGISTRATION_FAIL_REASON and no outputs. executions/search returns no row for it, as when a run fails before data platform records it.
• An SDF record named MOCK_POSE_INGESTION_FAIL_NAME lets the run complete while its data-platform row reads Failed.
• An SDF record named MOCK_POSE_REJECTED_NAME stays in jobOutputs but is left out of result-explorer while the run completes. This mimics the platform rejecting one record during storage and drives the short-count error.
• A synchronous import-dataset run whose inputs include result_lane is rejected with HTTP 400 unless the requested tool version is at least IMPORT_DATASET_RESULT_LANE_MIN_VERSION (src/utils/constants.py). This mirrors older import-dataset input schemas, which reject unknown keys. Pose registration sends result_lane only when the pinned version accepts it.

Running the Mock Server¶

Using in Tests¶

Pass --env local to pytest:

uv run pytest --env local

This starts a MockServer on port 4931 for the duration of the test session (see conftest.py). With DO_ENV=local, DeepOriginClient defaults to http://127.0.0.1:4931 (same as the mock).

Full platform locally (gateway on 6010): set DO_BASE_URL=http://127.0.0.1:6010 (and keep or set DO_ENV=local if inferred from URL). See LOCAL_ENDPOINT_GATEWAY in src/utils/constants.py.

The same tests can also run against a real environment:

uv run pytest --env dev

When --env dev is used, the mock server is not started and all requests go to the real platform API. This dual-mode design means the mock server must produce responses that are structurally identical to the real API — it is not a shortcut that skips validation.

Live test project: level-1 and integration tests against dev/staging/prod resolve a shared project by display name (do-dd-client-tests, see tests/integration_project.py). The canonical id is fetched from the platform at session start (create-if-missing via projects.create); it is not hard-coded. Set DO_PROJECT_ID to pin a different project without renaming. Local tests keep using the mock-only id 09DEFAULTPROJECT00.

Standalone Script¶

To run the mock server standalone for local development:

python tests/run_mock_server.py [PORT]

Where PORT is the port number to run the server on (default: 4931 via make mock-server).

Optional flags:

uv run python -m tests.run_mock_server --port 4931 --abfe-duration 30 --rbfe-duration 30

RBFE progress tree (local)¶

While an RBFE execution is Running, the mock synthesizes a v2 progress tree (matching dev's progressReport shape) instead of the legacy {"complete": N} bar. Stages follow platform-toolbox/tools/rbfe/workflow/workflow.yaml:

1. prepare-inputs → run-konnektor (Skipped when absent from steps) → build-pair-list → pair-pipeline per pair
2. Inside each pair: system-prep-task (Skipped for steps=["rbfe"]) → resolve-prepared-system → rbfe-e2e-task

Children pop up as the mock run advances. The simulation leaf ramps toolProgress.complete from 0 to 100 (not the legacy top-level complete key). Pace the run with --rbfe-duration on run_mock_server (default 30s for standalone make mock-server; pytest --env local keeps 5s for speed).

Implementation: build_rbfe_progress_tree() and _get_rbfe_progress_report() in tests/mock_server/routers/tools.py.

RBFE notebook (local)¶

The working RBFE notebook is docs/notebooks/clean/rbfe-1-single-pair.ipynb. It runs against the mock when DO_ENV=local and the server is up:

make mock-server   # port 4931

Set DO_ENV=local in .env (and JOB_WATCH_BLOCK=1 for nbconvert --execute), then open the notebook. Fixtures are under:

Integration coverage: tests/test_rbfe_local.py (uv run pytest tests/test_rbfe_local.py --env local).

How Tool Executions Work¶

This is the most complex part of the mock server. When client code runs a tool (e.g., Docking(protein=..., pocket=..., ligand=...).run(), PocketFinder(protein).run(), or Molprops(...).run()), the following happens end-to-end. All runnable work goes through client.executions.create() and the mock's POST /tools/{org}/tools/{tool_key}/{tool_version}/executions route (run_tool in routers/tools.py).

1. Building the execution DTO¶

run_tool inspects the request body and the URL's tool_key and produces a tool execution DTO with executionId, status, userInputs, jobOutputs, quotationResult, tool: {key, version}, and type: "ToolExecution". Several tool keys are special-cased:

• deeporigin.docking (single ligand, sync=True), deeporigin.pocket-finder, deeporigin.system-prep, deeporigin.protein-prep complete in one POST via _create_blocking_run_dto and inject their fixture-backed jobOutputs into the result-explorer pool. Protein Prep v2 branches on inputs.action: recommend loads tool-runs/deeporigin.protein-prep/recommend.json; prepare (or omitted action) loads run.json.
• deeporigin.mol-props-protonation returns deterministic protonation_states from the request inputs (_build_protonation_execution).
• deeporigin.mol-props-* (logd, logp, ames, …) hash the normalized request body and look up the recorded outputs at tests/fixtures/tool-runs/{tool_key}/{body_hash}.json (_build_molprops_execution). The fixture's per-ligand jobOutputs rows are re-keyed by the request's ligand IDs.
• deeporigin.mol-props-combined (Molprops) synthesizes per-ligand rows and a matching quotationResult (_build_combined_molprops_execution). When the request body includes approveAmount: -1 (quote-only; <= 0 also accepted), the mock clears jobOutputs and sets status to Quoted.
• deeporigin.bulk-docking (approveAmount<=0) loads tests/fixtures/tool-runs/deeporigin.bulk-docking/quote.json and scales the quotation by ligand count.
• All other tool keys fall back to a generic Quoted execution DTO via _create_execution_dto.

2. Request hashing for molprops fixtures¶

For deeporigin.mol-props-*, _build_molprops_execution normalizes the request body with normalize_tool_execution_body (strips IDs, clusterId, tag, app, session) and hashes it to find a fixture:

tests/fixtures/tool-runs/{tool_key}/{body_hash}.json

Because IDs are stripped before hashing, the same fixture handles different ligand IDs; the per-row ligand_id is patched in afterwards from the request's ligand inputs.

3. ID replacement in tool outputs¶

Some fixtures (docking, pocket-finder, system-prep) were recorded with whatever protein/ligand IDs existed at recording time, but in tests entities get fresh IDs every run. _replace_ids_in_outputs recursively walks the fixture's jobOutputs (or the legacy functionOutputs field on older fixtures) and rewrites any protein_id / ligand_id / ligand1_id value to match the IDs from the current request's userInputs.

4. Result-explorer injection¶

In production, when a tool execution completes, a message-queue flow writes structured outputs (pockets, poses, system, …) into a result-explorer table. Client code later queries this table to retrieve results — for example, Docking.get_results() calls client.results.get_poses(compute_job_id=...), which searches the result-explorer.

The mock server emulates this with _inject_result_explorer_records_from_outputs. After a tool execution returns, it checks the output_key_map:

output_key_map = {
    "deeporigin.pocketfinder": ("pockets", "pocket"),
    "deeporigin.pocket-finder": ("pockets", "pocket"),
    "deeporigin.docking": ("poses", "pose"),
    "deeporigin.system-prep": ("system", "preparedsystem"),
    "deeporigin.protein-prep": ("protein", "preparedprotein"),
}

Each entry maps a tool manifest key to the jobOutputs field that should be mirrored into the shared result-explorer store and the result_type to tag those records with. Tool-specific helpers (_inject_docking_tool_execution_results, _inject_pocketfinder_tool_execution_results, _inject_sysprep_tool_execution_results, _inject_protein_prep_tool_execution_results) load the same fixture shapes and call this generic injector.

When adding a new tool, extend output_key_map and any tool-specific injectors in routers/tools.py so downstream queries (e.g., Pocket.from_result, Docking.get_results()) can find the records.

5. Result-explorer queries¶

When client code queries results (e.g., client.results.get_poses(compute_job_id=...)), it hits the data-platform router's result-explorer/search endpoint. This applies _apply_eq_filters, which checks filter values against both top-level record fields and nested data fields. Since the injected records store tool outputs under data, a filter like protein_id: {eq: "abc123"} matches record["data"]["protein_id"].

End-to-end example: docking¶

Here's the full flow for test_docking_run_lv2 in tests/test_docking.py:

1. registered_protein fixture
   → POST /data-platform/{org}/proteins  (creates protein, gets fresh ID)

2. registered_ligand fixture
   → POST /data-platform/{org}/ligands   (creates or syncs ligand)

3. Docking(protein=..., pocket=..., ligand=...).run()
   → POST /tools/{org}/tools/deeporigin.docking/{version}/executions
   → run_tool:
       a. build a Completed execution DTO via _create_blocking_run_dto
       b. _inject_docking_tool_execution_results loads the docking fixture,
          replaces protein_id/ligand_id, and pushes the "poses" rows into
          the result-explorer store

4. run() → get_results() → client.results.get_poses(compute_job_id=...)
   → POST /data-platform/{org}/result-explorer/search
       filter: {compute_job_id: {eq: "<executionId>"}, ...}
   → _apply_eq_filters matches the injected rows
   → returns the poses injected in step 3b (falls back to jobOutputs.poses)

Entity Stores and Fixtures¶

In-memory stores¶

The mock server maintains in-memory dicts for proteins and ligands. These start empty for proteins (fresh registration every test run) and pre-populated for ligands (from SDF files and JSON fixtures).

Ligand pre-population¶

_load_ligand_fixtures scans multiple sources on startup:

1. src/data/brd/*.sdf — package-data ligands
2. tests/fixtures/ligands-brd-all.sdf, 42-ligands.sdf, brd-7.sdf
3. tests/fixtures/ligand_*.json

Ligand IDs are deterministic — derived from a SHA-256 hash of the canonical SMILES. This means the same ligand always gets the same ID across test runs, which is important for fixture matching.

Result-explorer pre-population¶

_load_result_explorer_fixtures loads any tests/fixtures/result-explorer-*.json files into the shared results store at startup. Tool executions then append to this same store at runtime.

Configuring Your Client¶

To use the mock server with your code, configure the DeepOriginClient to point to the mock server URL:

from deeporigin.platform.client import DeepOriginClient

client = DeepOriginClient(
    token="test-token",  # Any token works with the mock server
    org_key="deeporigin",  # Use any org_key
    base_url="http://127.0.0.1:4931",  # Mock server URL (port 4931 for tests)
    env="local",
)

Available Endpoints¶

The mock server implements the following endpoints:

• Files API: List, upload, download, and delete files. Listing a folder that isn't in tests/fixtures/files returns the files uploaded under it during the session (e.g. imports/staging).
• Tools API: List tools and tool definitions
• Executions API: List, get, cancel, confirm, and create tool executions
• Clusters API: List available clusters
• Entities API: Entity management (delete)
• Data Platform API: Entity CRUD (proteins, ligands), result-explorer search
• Billing API: Billing endpoints
• SSE stream: Gateway project stream that marks requested filters dirty once, then closes
• Health Check: /health endpoint

Extending the Mock Server¶

Adding a new tool¶

1. Record a fixture by running the tool against --env dev and saving the response as tests/fixtures/tool-runs/{tool_key}/{body_hash}.json (or run.json / quote.json for tools that don't hash by request body).
2. If the tool produces structured outputs that need to be queryable via result-explorer, add an entry to output_key_map in _inject_result_explorer_records_from_outputs (tests/mock_server/routers/tools.py) and a small injector helper for that tool.
3. Run the test with --env local to verify.

Adding a new entity endpoint¶

1. Add a route handler to the appropriate router in tests/mock_server/routers/
2. Optionally add fixture files in tests/fixtures/ if you need realistic data
3. Use load_fixture() to load JSON fixtures

Adding a new router¶

1. Create a new file in tests/mock_server/routers/
2. Define a create_*_router() factory function that accepts shared state
3. Include the router in MockServer._setup_routes() in server.py

Limitations¶

• Authentication is not validated (any token works)
• File storage is in-memory and lost when the server stops
• Tool executions return pre-recorded fixtures, not computed results
• Rate limiting and other production features are not implemented

For production use, always use the real DeepOrigin Platform API.