JSON-delta: a diff/patch pair for JSON-serialized data structures

JSON-delta is a multi-language software suite for computing deltas between JSON-serialized data structures, and applying those deltas as patches. It enables separate programs at either end of a communications channel (e.g. client and server over HTTP, or two processes using IPC) to manipulate the same data structure while minimizing communications overhead.

If you’re not clear what this means, or you simply can’t see the point, see JSON-delta by example for a (somewhat whimsical) exposition of the basic idea.

If, on the other hand, you’re fully sold and want to know how to use one of the available implementations of JSON-delta, see The JSON-delta API, which documents the functions every implementation makes available. This is something like a “standard” for the JSON-delta “API” (I’ll consider that the software has outgrown these quotes when it gets more users than just me…) Since the Python implementation is the most developed, if you’re thinking in terms of standards, you can consider it the reference implementation.

Further documentation of individual implementations is also available, along with manpages for the CLI programs json_diff(1) and json_patch(1)

Donations to support the continuing development of JSON-delta will be gratefully received via gratipay, PayPal (himself@phil-roberts.name) or Bitcoin: 1HPJHRpVSm1Y4zrgppd2c6LysjxeabbQN4

News

  • Bugfix release 1.1.3 is out. Serious bugs are addressed in this release, so Javascript users and anyone who relies on the upatch functionality should upgrade.
  • Bugfix release 1.1.2 is out, featuring a sharper distinction between minimal and non-minimal diffs. Non-minimal diffs can now be called for on the command line by running json_diff --fast.
  • Bugfix release 1.1.1 is out. Javascript users and anyone who uses udiffs should upgrade.
  • The Heisenbug referred to below has been fixed, along with a more serious bug: in v1.0 of both the Python and Javascript implementations, more than one addition to a non-top-level array—that is, an array nested within one or more other arrays or objects—is not encoded properly in diffs where the minimal flag is set to True. There was no official release of v1.0 for Python, but for ease of reference I’m calling both the fixed versions 1.1. It is recommended that all users upgrade.
  • There is a Heisenbug in the udiff part of what I’m retroactively calling v1.0 of the python implementation: it shows up some of the time for one test case, and then only when running python3. Due to this and other bugfixes, I recommed upgrading to v1.1.
  • v1.0 of the Javascript implementation has now been released. For JSON-format diffs it is now every bit as capable as the python version, and it has been extensively tested, not only against the JSON-delta test suite, but also with JSlint and JShint.

Downloads

The Python implementation (compatible with version 2.7 and later, including 3) is available from PyPI.

You can download the Javascript versions here (full, minified)

(Please, for the sake of my bandwidth bills, serve your own copy rather than hot-linking mine!)

The racket and perl implementations are alpha-quality at best. If you want to check them out, I recommend looking at the source repo (no pun intended): development of JSON-delta takes place against a master repository containing all implementations.

Implementations

The following table summarizes the feature-completeness of the available implementations of JSON-delta, with links to implementation-specific notes for each:

Language Patch Diff Compact U-patch U-diff
Python (2.7 and newer)
Javascript
Racket

The features described are as follows:

Patch
The implementation can manipulate data structures according to a diff in the format specified above.
Diff
The implementation can calculate deltas between two data structures in the format specified above.
Compact
Diffs produced by the implementation are as small as I can possibly make them, using a variant of Needleman-Wunsch sequence alignment to optimize stanzas modifying JSON arrays.
U-diff
The implementation is capable of emitting diffs in a format reminiscent of the output of diff -u, which is designed to be more human-readable than the JSON format, to facilitate debugging.
U-patch
The implementation can apply U-format patches.

The beginnings of a patch implementation in Perl can also be found in the source repo, but, because Perl doesn’t have as ramified a type ontology as Javascript, numeric values do not round-trip cleanly, so work on the Perl implementation is stalled for now.

Indices and tables