Functional to mapping in the preferred language
  • Python 66.9%
  • TypeScript 22%
  • Rust 4.2%
  • JavaScript 3.8%
  • CSS 1.7%
  • Other 1.4%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Giacomo 'Mr. Wolf' Furlan 1f45d75ed5
All checks were successful
CI / Offline tests (push) Successful in 1m43s
CI / Git fallback (gitoxide) (push) Successful in 2m6s
CI / Integration tests (push) Successful in 2m14s
New line after anchor to avoid messing with markdown
2026-11-03 12:35:11 +01:00
.forgejo/workflows Merge branch 'main' of ssh://git.giacomofurlan.name:2200/elegos/json2json-mapping-engine 2026-11-03 10:57:14 +01:00
.github/workflows Merge branch 'main' of ssh://git.giacomofurlan.name:2200/elegos/json2json-mapping-engine 2026-11-03 10:57:14 +01:00
build/compiled Groups: lists exist only under their guard, loops with filters and oneOf roots build; import skill and assemble fixes 2026-10-12 15:28:18 +02:00
cli Import runs: mapper is called alone, without pipes into other programs 2026-10-15 16:22:09 +02:00
compiler Engine gaps, step 5b: a loop over the parts of a text (split_list); Python loop filters read a missing list as empty 2026-10-15 14:53:53 +02:00
desktop Guide pictures regenerated on mocked states; new ones for loops, suggestions, validations and the import; importing page; the tours as pages 2026-10-16 08:04:34 +02:00
dev_scripts Release workflows: one job per target (linux, windows, macos), then the release job; one cache volume per target 2026-10-06 15:50:34 +02:00
docs Merge branch 'main' of ssh://git.giacomofurlan.name:2200/elegos/json2json-mapping-engine 2026-11-03 10:57:14 +01:00
fn feat: bootstrap project scaffolding — uv/ruff/basedpyright, mapper.toml loader, CI 2026-09-24 12:06:28 +02:00
generated Golden mapping, schemas and tests for the validation of input and output 2026-10-12 10:54:50 +02:00
mappings Groups: lists exist only under their guard, loops with filters and oneOf roots build; import skill and assemble fixes 2026-10-12 15:28:18 +02:00
packaging Installer: the dependencies ship as byte-code only, about half the files 2026-10-16 07:25:59 +02:00
runtime feat: JSON Schema → generated stubs and dataclasses (Schema Generator, M1) 2026-09-24 12:06:38 +02:00
schemas Golden mapping, schemas and tests for the validation of input and output 2026-10-12 10:54:50 +02:00
skills/import-functional-spec Import runs: mapper is called alone, without pipes into other programs 2026-10-15 16:22:09 +02:00
tests Completion: comprehension variables, a loop's own element, order by keys, sorted in the developer dialect 2026-10-16 07:36:29 +02:00
transpiler Completion: comprehension variables, a loop's own element, order by keys, sorted in the developer dialect 2026-10-16 07:36:29 +02:00
.gitignore Desktop packages built from Linux (AppImage, Windows NSIS installer and portable zip, macOS dmg), draft releases from v* tags with the version and notes taken from the tag and CHANGELOG.md, window title J2JME v<version> (<commit>), Wayland icon matched to the integrated .desktop 2026-10-05 22:25:32 +02:00
CHANGELOG.md Guide pictures regenerated on mocked states; new ones for loops, suggestions, validations and the import; importing page; the tours as pages 2026-10-16 08:04:34 +02:00
LICENSE Add GPLv3 license with generated-output exception, and surface it in README, CLI, and desktop help 2026-10-07 09:09:14 +02:00
mapper.toml Engine version in mapper.toml: newer tool updates the project after a warning, older tool refuses unless the risk is accepted, never moved back; mapper upgrade, install-mapper.sh reads it, manifest no longer carries it 2026-10-05 23:31:04 +02:00
pyproject.toml Import: read functional documents, build and run import plans, add the import skill 2026-10-12 10:54:50 +02:00
README.md New line after anchor to avoid messing with markdown 2026-11-03 12:35:11 +01:00
uv.lock Import: read functional documents, build and run import plans, add the import skill 2026-10-12 10:54:50 +02:00

JSON 2 JSON mapping engine logo json2json-mapping-engine

Latest release CI (latest commit) Python 3.14+ Status: work in progress

Target: PL/SQL Target: Python

A compiled, hard-typed JSON-to-JSON mapping engine designed for polyglot enterprise runtimes. Write declarative, target-agnostic mapping rules; the compiler validates them against strict JSON Schemas and transpiles them into optimized, native artifacts for any supported backend.

Note

No runtime interpreters. No framework overhead. Just deterministic, production-ready native code tailored for your specific execution environment.

📑 Contents

🏛️ Why This Engine Exists

In modern enterprise architectures, data mapping between decoupled systems—ranging from legacy transactional databases to distributed big data pipelines—is a major source of technical debt, silent production failures, and performance bottlenecks.

Traditional approaches fall into two flawed categories:

  1. Runtime Interpretation (DSLs & Heavy Frameworks): Interpreted mapping engines bypass static code analysis, introduce compute overhead, and delay error discovery until the code hits live production under heavy load.
  2. Inconsistent Hand-Written Code: Manual implementation leads to fragmented logic, security vulnerabilities, and unmaintainable variations across repositories, especially when AI code-generation is used without strict boundaries.

json2json-mapping-engine solves this by decoupling the mapping design from the execution runtime. It moves 100% of the type-checking and validation complexity to the compile-time layer. If a mapping rule compiles, it is mathematically guaranteed to be consistent with your schemas.


🎯 Supported & Upcoming Backends

The engine architecture separates the front-end (rule transpilation and schema verification) from the back-end (code generation). This allows it to target entirely different computing paradigms with the same set of rules:

  • Python (✅ Complete): Emits clean, dependency-free Python modules leveraging native data structures.
  • PL/SQL / Oracle (✅ Complete): Emits optimized, high-performance database packages, eliminating the need for unsafe dynamic SQL evaluation or mid-tier roundtrips.
  • Java (⏳ Planned): Target for cloud-native microservices and robust backend ecosystems.
  • PySpark (⏳ Planned): Target for big data engineering, translating schema mappings into optimized DataFrame transformations for distributed clusters.

🧩 How it works

.fn.yaml ──► transpiler ──► .map.py ──┐
                                      ├──► compiler ──► target artifact (e.g. PL/SQL package)
JSON Schemas ──► stubs ───────────────┘

Everything up to the type checker is target-agnostic; only the backend emits code. Project settings live in mapper.toml.

🎯 Targets

Target State
PL/SQL (Oracle) ✅ reference implementation, see docs/backends/plsql.md
Python ✅ complete (generated modules need Python 3.10+)

Writing another one: docs/backends/writing-a-backend.md.

🖥️ Desktop tool

A Tauri app (Linux AppImage, Windows installer and portable zip, macOS dmg) to edit rules in a grid, try a mapping, run the checks and use git (commit, pull, push) without leaving it.

Note

Don't author files in bare text editors, do it in a feature-rich, hint-supported application!

See desktop/README.md.

Desktop tool

🧩 Code Generation: Before & After

Instead of interpreting rules at runtime, json2json-mapping-engine takes a declarative functional mapping and compiles it into the exact code a senior engineer would write manually.

Note

The following are mere examples, and might differ from the final result, depending on the engine version.

1. The Source Rule (orders.fn.yaml)

mapping:
  target: "CustomerOrder"
  rules:
    - field: "order_id"
      transform: "source.id.upper()"
    - field: "total_amount"
      transform: "source.price * source.quantity"

2. The Compiled Outputs (Zero Dependencies)

🔹 Target: PL/SQL (Optimized for Oracle Database)

CREATE OR REPLACE PACKAGE BODY pkg_orders_mapping AS
   PROCEDURE map_json(p_source IN CLOB, p_target OUT CLOB) IS
   BEGIN
      -- Pure native PL/SQL JSON features. No EXECUTE IMMEDIATE, no overhead.
      SELECT JSON_OBJECT(
                'order_id'     VALUE UPPER(JSON_VALUE(p_source, '$.id')),
                'total_amount' VALUE (TO_NUMBER(JSON_VALUE(p_source, '$.price')) * TO_NUMBER(JSON_VALUE(p_source, '$.quantity')))
             )
      INTO p_target FROM DUAL;
   END map_json;
END pkg_orders_mapping;

🔹 Target: Python (Clean & Type-Hinted)

def map_json(source: dict) -> dict:
    # Pure native Python, fully compliant with your target JSON Schema
    return {
        "order_id": str(source["id"]).upper(),
        "total_amount": float(source["price"]) * int(source["quantity"])
    }

📦 Install

uv sync --all-extras --dev
uv run mapper --help
  • 🖥️ Desktop tool from source: it is the app, not mapper, that people work with. On top of the Python setup above it needs:
    • Rust (stable) and a C compiler;
    • Node.js and npm (the React UI);
    • cargo install tauri-cli --version '^2';
    • on Linux, the WebKitGTK 4.1 dev packages (Fedora: webkit2gtk4.1-devel).
(cd desktop/ui && npm install)
dev_scripts/start_dev.sh      # the app in dev mode, with the settings Wayland needs

Details in desktop/README.md.

🚀 Quick start

mapper build      # transpile (driven mode) + compile into build/compiled/
mapper check      # regenerate and fail if it differs from the committed files
mapper publish    # check, build, then publish the files and manifest.json
mapper verify     # check a published folder against its manifest.json

Try it on the sample project in desktop/fixtures/orders-project.

🤖 CI

mapper check and mapper build need no Python on the runner: packaging/install-mapper.sh installs the engine version pinned in mapper.toml. Works on Forgejo Actions and GitHub Actions: docs/CI.md.

🛠️ Development

uv run ruff check .
uv run basedpyright
uv run mapper test --integration   # needs an Oracle database

Both pipelines (.forgejo/workflows and .github/workflows) are kept in sync.

📚 Documentation

📜 Open Source License & The Commercial Exception

Copyright (C) 2026 Giacomo Furlan.

This engine is proudly licensed under the GNU General Public License v3.0 (GPLv3). However, to make it fully viable for commercial and enterprise production ecosystems, it features a crucial legal boundary:

⚠️ The Generated Output Exception

Note

Everything the engine generates from your mapping rules — including packages, modules, classes, functions, scripts, etc — is completely EXEMPT from the GPLv3 constraints.

  • Your Rules, Your Code: The output generated by the compiler belongs entirely to you. You can distribute it, keep it proprietary, or relicense it under any terms of your choosing (including closed-source commercial applications).
  • What stays GPLv3: Only the core engine itself (the compiler, the transpiler, the CLI tool, and the Desktop Studio) is protected by the GPLv3 to ensure that improvements to the tool chain remain open to the community.

For the exact legal phrasing of this exception, please refer to the end of the LICENSE file.