KeyMap Utilities

Extras that are useful when using NestedText.

nestedtext.get_keys(keys, keymap, *, original=True, strict=True, sep=None)[source]

Returns a key sequence given a normalized key sequence.

Keys in the dataset output by the load functions are referred to as normalized keys, even though no key normalization may have occurred. This distinguishes them from the original keys, which are the keys given in the NestedText document read by the load functions. The original keys are mapped to normalized keys by the normalize_key argument to the load function. If normalization is not performed, the normalized keys are the same as the original keys.

By default this function returns the original key sequence that corresponds to keys, a normalized key sequence.

Parameters:

  • keys – The sequence of normalized keys that identify a value in the dataset.

  • keymap – The keymap returned from load() or loads().

  • original – If true, return keys as originally given in the NestedText document (pre-normalization). Otherwise return keys as they exist in the dataset (post-normalization). The value of this argument has no effect if the keys were not normalized.

  • strict –

    strict controls what happens if the given keys are not found in keymap.

    The various options can be helpful when reporting errors on key sequences that do not exist in the data set. Since they are not in the dataset, the original keys are not available.

    True or “error”:

    A KeyError is raised.

    False or “all”:

    All keys given in keys are returned.

    ”found”:

    Only the initial available keys are returned.

    ”missing”:

    Only the missing final keys are returned.

    When returning keys, the initial available keys are converted to their original form if original is true, The missing keys are always returned as given.

  • sep – A join string. If given the keys are interleaved with sep and joined into a string before being returned.

Returns:

A tuple containing the desired keys if sep is not given. A string formed by joining the keys with sep if sep is given.

Examples

>>> import nestedtext as nt

>>> contents = """
... Names:
...     Given: Fumiko
... """

>>> def normalize_key(key, keys):
...     return key.lower()

>>> data = nt.loads(contents, "dict", normalize_key=normalize_key, keymap=(keymap:={}))

>>> print(get_keys(("names", "given"), keymap))
('Names', 'Given')

>>> print(get_keys(("names", "given"), keymap, sep="❭"))
Names❭Given

>>> print(get_keys(("names", "given"), keymap, original=False))
('names', 'given')

>>> keys = get_keys(("names", "surname"), keymap, strict=True)
Traceback (most recent call last):
...
KeyError: ('names', 'surname')

>>> print(get_keys(("names", "surname"), keymap, strict="found"))
('Names',)

>>> print(get_keys(("names", "surname"), keymap, strict="missing"))
('surname',)

>>> print(get_keys(("names", "surname"), keymap, strict="all"))
('Names', 'surname')
nestedtext.get_value(data, keys)[source]

Get value from keys.

Parameters:

  • data – Your data set as returned by load() or loads().

  • keys – The sequence of normalized keys that identify a value in the dataset.

Returns:

The value that corresponds to a tuple of keys from a keymap.

Examples

>>> import nestedtext as nt

>>> contents = """
... names:
...     given: Fumiko
...     surname: Purvis
... """

>>> data = nt.loads(contents, "dict")

>>> nt.get_value(data, ("names", "given"))
'Fumiko'
nestedtext.get_line_numbers(keys, keymap, kind='value', *, strict=True, sep=None)[source]

Get line numbers from normalized key sequence.

This function returns the line numbers of the key or value selected by keys. It is used when reporting an error in a value that is possibly a multiline string. If the location contained in a keymap were used the user would only see the line number of the first line, which may confuse some users into believing the error is actually contained in the first line. Using this function gives both the starting and ending line number so the user focuses on the whole string and not just the first line. This only happens for multiline keys and multiline strings.

If sep is given, either one line number or both the beginning and ending line numbers are returned, joined with the separator. In this case the line numbers start from line 1.

If sep is not given, the line numbers are returned as a tuple containing a pair of integers that is tailored to be suitable to be arguments to the Python slice function (see example). The beginning line number and 1 plus the ending line number is returned as a tuple. In this case the line numbers start at 0.

If keys corresponds to a composite value (a dictionary or list), the line on which it ends cannot be easily determined, so the value is treated as if it consists of a single line. This is considered tolerable as it is expected that this function is primarily used to return the line number of leaf values, which are always strings.

Parameters:

  • keys – The sequence of normalized keys that identify a value in the dataset.

  • keymap – The keymap returned from load() or loads().

  • kind – Specify either “key” or “value” depending on which token is desired.

  • strict – If strict is true, a KeyError is raised if keys is not found. Otherwise the line number that corresponds to composite value that would contain keys if it existed. This composite value corresponds to the largest sequence of keys that does actually exist in the dataset.

  • sep – The separator string. If given a string is returned and sep is inserted between two line numbers. Otherwise a tuple is returned.

Raises:

KeyError – If keys are not in keymap and strict is true.

Example

>>> import nestedtext as nt

>>> doc = """
... key:
...     > this is line 1
...     > this is line 2
...     > this is line 3
... """

>>> data = nt.loads(doc, keymap=(keymap:={}))
>>> keys = ("key",)
>>> lines = nt.get_line_numbers(keys, keymap, sep="-")
>>> text = doc.splitlines()
>>> print(
...     f"Lines {lines}:",
...     *text[slice(*nt.get_line_numbers(keys, keymap))],
...     sep="\n"
... )
Lines 3-5:
    > this is line 1
    > this is line 2
    > this is line 3
nestedtext.get_location(keys, keymap)[source]

Returns Location information from the keys. None is returned if location is unknown.

Parameters:

  • keys – The sequence of normalized keys that identify a value in the dataset.

  • keymap – The keymap returned from load() or loads().

nestedtext.annotate(keys, keymap, *, key_leading=(), key_trailing=(), value_leading=(), value_trailing=(), header=(), footer=(), spacing=None)[source]

Create or update keymap[tuple(keys)] with comments and per-Location spacing in a single call.

This is the from-scratch counterpart to the keymap that load() builds. Each of the four per-key slot kwargs – key_leading, key_trailing, value_leading, value_trailing – accepts either:

  • an iterable of Comment objects (static), which become the comments attached to this Location at that slot. Each Comment is interpreted in tab mode: its tab field (defaulting to 0 when None) is the tabstop offset from the slot’s natural indent, resolved by the dumper at emit time using the dumps(indent=...) setting; or

  • a callable (a provider) with the signature

    provider(child_key) -> list[Comment]

    installed on this Location to be invoked by the dumper for each child of this Location’s value. The returned Comments are prepended to the matching child’s static comments at the same slot, before rendering. This is how dynamic section / group headers are produced (the closure can dedup over previously-seen keys). Comments returned by a provider with tab=None are normalized to tab=0 at emit time. Providers are not JSON-serializable and are dropped on keymap_to_jsonable() round-trips.

The natural indent for each slot, given N = len(keys) and S = dumps.indent:

slot

natural indent

key_leading

(N - 1) * S

key_trailing, value_leading, value_trailing

N * S

header, footer

0

Static lists in the per-key slots are not allowed at the root (keys == ()) since the root has no key line to attach to. A provider callable, however, is allowed at the root – it decorates each top-level child.

spacing, if given, is applied via Location.set_spacing().

Parameters:

  • keys – The keys tuple identifying the Location. Use () for the document-root Location.

  • keymap – The keymap dict to mutate.

  • key_leading – Either an iterable of Comment objects (stored on this Location) or a callable provider (invoked per child of this Location; see above). Static lists are not allowed when keys == ().

  • key_trailing – Either an iterable of Comment objects (stored on this Location) or a callable provider (invoked per child of this Location; see above). Static lists are not allowed when keys == ().

  • value_leading – Either an iterable of Comment objects (stored on this Location) or a callable provider (invoked per child of this Location; see above). Static lists are not allowed when keys == ().

  • value_trailing – Either an iterable of Comment objects (stored on this Location) or a callable provider (invoked per child of this Location; see above). Static lists are not allowed when keys == ().

  • header – Iterables of Comment objects for the document header and footer. Only allowed when keys == ().

  • footer – Iterables of Comment objects for the document header and footer. Only allowed when keys == ().

  • spacing – Per-Location spacing dict; see Location.set_spacing().

Returns:

The Location that was created or updated.

Raises:

ValueError – if a static list is supplied for any of key_leading/key_trailing/value_leading/ value_trailing at the root, or if header/footer is supplied for non-root keys.

Transferring a Keymap Between Processes

When the load → modify → dump cycle is split across processes (so the data is serialized between them, for example as JSON), the keymap must travel with the data if comments and original key spellings are to be restored on dump. keymap_to_jsonable() reduces a keymap to a structure built from plain dict, list, str, int, and None — pass it through json, msgpack, or any other encoder of your choice — and keymap_from_jsonable() rebuilds the keymap on the other side. Only the information dump() actually consults is preserved (original key strings and comment slots); source line and column numbers are discarded.

nestedtext.keymap_to_jsonable(keymap, **kwargs)[source]

Reduce a keymap to a JSON-serializable structure for use with dumps().

Captures only what dumps() needs from the keymap to reconstruct the original file: the original key strings (so map_keys can restore them) and the per-entry comment slots, plus the document header / footer on keymap[()]. Source line/column information is discarded. Per-slot provider callables (set via Location.set_key_leading_provider() and the matching set_*_provider methods) are also dropped because callables are not JSON-serializable; rebuilt keymaps therefore omit any provider-driven decoration.

The returned object is built from dict, list, str, int, and None — safe to pass through json, msgpack, or any similar encoder.

Parameters:

  • keymap – The keymap returned from load() or loads(), or any equivalent mapping from key-tuples to Location objects.

  • **kwargs – Any extra keyword arguments are included in the returned structure under a top-level “meta” key. These values are not used by keymap_from_jsonable() but are included to allow you to include any extra metadata you wish in the JSON-serializable structure. No attempt is made to ensure that the values in kwargs are themselves JSON-serializable, so you should ensure that they are if you intend to pass the output through a JSON encoder.

Returns:

A JSON-serializable dict. Pass it to keymap_from_jsonable() to rebuild a keymap that can be given to dumps() as map_keys=.

nestedtext.keymap_from_jsonable(data)[source]

Rebuild a keymap from the output of keymap_to_jsonable().

The returned mapping is suitable for passing to dumps() (or dump()) as map_keys=; it will restore the original key strings and inject the captured comments. Locations in the rebuilt keymap do not carry source line/column information.

Parameters:

data – The JSON-serializable structure produced by keymap_to_jsonable() (or an equivalent reconstruction of it, e.g., from json.loads).