Usage

Installation and requirements

To install capnpy, just type:

$ pip install capnpy

capnpy relies on the official capnproto implementation to parse the schema files, so it needs to be able to find the capnp executable to compile a schema. It requires capnp 0.5.0 or later.

Quick example

Suppose to have a capnp schema called example.capnp:

@0xe62e66ea90a396da;
struct Point {
    x @0 :Int64;
    y @1 :Int64;
}

You can use capnpy to read and write messages of type Point:

import capnpy
# load the schema using dynamic loading
example = capnpy.load_schema('example')
# create a new Point object
p = example.Point(x=1, y=2)
# serialize the message and load it back
message = p.dumps()
p2 = example.Point.loads(message)
print('p2.x ==', p2.x)
print('p2.y ==', p2.y)
p2.x == 1
p2.y == 2

Compiling schemas

capnpy supports different ways of compiling schemas:

setuptools integration
to compile and distribute schemas using setup.py.
Dynamic loading
to compile and load capnproto schemas on the fly.
Manual compilation
to compile schemas manually.

If you use setup.py or manual compilation, you need capnp to compile the schema, but not to load it later; this means that you can distribute the precompiled schemas, and the client machines will be able to load it without having to install the official capnproto distribution.

If you use dynamic loading, you always need the capnp executable whenever you want to load a schema.

Integration with setuptools

If you use setuptools, you can use the capnpy_schema keyword to automatically compile your schemas from setup.py:

from setuptools import setup
setup(name='foo',
      version='0.1',
      packages=['mypkg'],
      capnpy_schemas=['mypkg/example.capnp'],
      )

You can specify additional compilation options by using capnpy_options:

from setuptools import setup
setup(name='foo',
      version='0.1',
      packages=['mypkg'],
      capnpy_options={
          'pyx': False,          # do NOT use Cython (default is 'auto')
          'convert_case': False, # do NOT convert camelCase to camel_case
                                 # (default is True)
      }
      capnpy_schemas=['mypkg/example.capnp'],
      )

Manual compilation

You can manually compile a capnproto schema by using python -m capnpy compile:

$ python -m capnpy compile example.capnp

This will produce example.py (if you are using py mode) or example.so (if you are using pyx mode). Run python -m capnpy --help for additional options.

Dynamic loading

To dynamically load a capnproto schema, use capnpy.load_schema; its full signature is:

def load_schema(modname=None, importname=None, filename=None,
                pyx='auto', options=None):
    ...

modname, importname and filename corresponds to three different ways to specify and locate the schema file to load. You need to pass exactly one of them.

modname (the default) is interpreted as if it were the name of a Python module with the .capnp extension. This means that it is searched in all the directories listed in sys.path and that you can use dotted names to load a schema inside packages or subpackages:

>>> import capnpy
>>> import mypackage
>>> mypackage
<module 'mypackage' from '/tmp/mypackage/__init__.pyc'>
>>> example = capnpy.load_schema('mypackage.mysub.example')
>>> example
<module 'example' from '/tmp/mypackage/mysub/example.capnp'>

This is handy because it allows you to distribute the capnproto schemas along the Python packages, and to load them with no need to care where they are on the filesystem, as long as the package is importable by Python.

importname is similar to modname, with the difference that it uses the same syntax you would use in capnproto’s import expressions. In particular, if you use an absolute path, load_schema searches for the file in each of the search path directories, which by default correspond to the ones listed in sys.path. Thus, the example above is completely equivalent to this:

>>> example = capnpy.load_schema(importname='/mypackage/mysub/example.capnp')
>>> example
<module 'example' from '/tmp/mypackage/mysub/example.capnp'>

Finally, filename specifies the exact file name of the schema file. No search will be performed.

pyx specifies whether to use pyx or py mode. options can be used to change the default compilation options:

>>> from capnpy.annotate import Options
>>> example = capnpy.load_schema('example', options=Options(convert_case=False))

Compilation options

The capnpy schema compiler has two modes of compilation:

py mode
Generate pure Python modules, which can be used either on CPython or PyPy: it is optimized to be super fast on PyPy. It produces slow code on CPython, but it has the advantage of not requiring cython. This is the default on PyPy.
pyx mode
Generate pyx modules, which are then compiled into native extension modules by cython and gcc. It is optimized for speed on CPython. This is the default on CPython, if cython is available.

Moreover, it supports the following options:

version_check
If enabled, the compiled schema contains a check which is run at import time to ensure that the current version of capnpy matches to the one we compiled the schema with. See note below for more details. The default is True.
convert_case
If enabled, capnpy will automatically convert field names from camelCase to underscore_delimiter: i.e., fooBar will become foo_bar. The default is True.
text_type
Can be bytes or unicode, Determines the default Python type for Text fields. The default is bytes.
include_reflection_data
If enabled, capnpy will embed Reflection data into the compiled schemas.

Note

Version checking is needed in particular if you are using pyx mode, which is the default on CPython. Capnproto struct are represented by Python classes which inherits from capnpy.struct_.Struct: in pyx mode, this is a Cython cdef class, and it has a certain C layout which depends on the number and type of its fields. If the C layout at compilation and import time don’t match, you risk segfault and/or misbehavior. Since the internal layout of classes might change between capnpy version, the version check prevents this risk.

Options annotation

capnpy options can also be configured by using the $Py.options annotation, which can be applied to file, struct and field nodes. The annotation recurively applies also to all the children nodes and can be used to override the options used by the parents.

This can be used to have a more granular control on how certain capnproto types are translated into Python. For example, you could use it to apply the convert_case option only to certain structs or fields:

@0x97a960ad8d4cf616;
using Py = import "/capnpy/annotate.capnp";

# don't convert the case by default
$Py.options(convertCase=false);

struct A {
    fieldOne @0 :Int64;
}

struct B $Py.options(convertCase=true) {
    fieldOne @0 :Int64;
    fieldTwo @1 :Int64;
    fieldThree @2 :Int64 $Py.options(convertCase=false);
}
>>> mod = capnpy.load_schema('example_options')
>>> mod.A.fieldOne
<property object at ...>
>>> mod.B.field_one
<property object at ...>
>>> mod.B.field_two
<property object at ...>
>>> mod.B.fieldThree
<property object at ...>

In the example above, A.fieldOne is not converted because of the file-level annotation. B.field_one and B.field_two are converted because the annotation on the struct overrides it. Finally, B.fieldThree overrides it again.

Note

Note the different spelling of options names: when you specify them in setup.py, they follow Python’s naming_convention and thus are spelled e.g. convert_case and text_type. However, when you specify them as annotation, the capnproto schema language mandates camelCase.

Loading and dumping messages

The API to read and write capnproto messages is inspired by the ones offered by pickle and json:

  • capnpy.load(f, payload_type): load a message from a file-like object
  • capnpy.loads(s, payload): load a message from a string
  • capnpy.load_all(f, payload_type): return a generator which yields all the messages from the given file-like object
  • capnpy.dump(obj): write a message to a file-like object
  • capnpy.dumps(obj): write a message to a string

For example:

>>> import capnpy
>>> example = capnpy.load_schema('example')
>>> p = example.Point(x=100, y=200)
>>> mybuf = capnpy.dumps(p)
>>> mybuf
'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00d\x00\x00\x00\x00\x00\x00\x00\xc8\x00\x00\x00\x00\x00\x00\x00'
>>> p2 = capnpy.loads(mybuf, example.Point)
>>> print(p2.x, p2.y)
100 200

Alternatively, you can call load/loads directly on the class, and dump/dumps directly on the objects:

>>> p = example.Point(x=100, y=200)
>>> mybuf = p.dumps()
>>> p2 = example.Point.loads(mybuf)
>>> print(p2.x, p2.y)
100 200

By default, dump and dumps try to use a fast path which is faster if you pass an object which is compact. If the fast path can be taken, it is approximately 5x faster on CPython and 10x faster on PyPy. However, if the object is not compact, the fast path check makes it ~2x slower. If you are sure that the object is not compact, you can disable the check by passing fastpath=False:

>>> mybuf = p.dumps(fastpath=False)

Loading from sockets

In case you want to load your messages from a socket, you can use capnpy.buffered.BufferedSocket to wrap it into a file-like object:

>>> from capnpy.buffered import BufferedSocket
>>> sock = socket.create_connection(('localhost', 5000))
>>> buf = BufferedSocket(sock)
>>> example.Point.load(buf)

Warning

The obvious solution to wrap a socket into a file-like object would be to use socket.makefile(). However, because of this bug it is horribly slow. Don’t use it. See also the benchmarks.

Raw dumps

Raw dumps are intented primarly for debugging and should never be used as a general transmission mechanism. They dump the internal state of the segments and the offsets used to identify a given capnproto object.

In particular, they dump the whole buffer in which the object is contained, which might be much larger that the object itself.

If you encounter a canpy bug, you can use _raw_dumps and _raw_loads to save the offending object to make it easier to reproduce the bug:

>>> p = example.Point(x=100, y=200)
>>> mydump = p._raw_dumps()
>>> p2 = example.Point._raw_loads(mydump)
>>> print(p2.x, p2.y)
100 200

capnproto types

Text

Capnproto defines Text fields as “always UTF-8 encoded and NUL-terminated”. There are at least two reasonable ways to represent this in Python:

  • as bytes: this will contain the undecoded UTF-8 string.
  • as unicode: this will automatically do .decode('utf-8') for you. However, it is potentially less efficient because capnpy needs to re-decode the string again and again any time you read the field.

By default, Text fields are represented as bytes. You can change the default behavior by setting the appropriate Compilation options. In case you are using Integration with setuptools, you need to pass capnpy_options={'text_type': 'unicode'} in your setup.py.

If you want more granular control, you can annotate single files/struct/fields by using the Options annotation.

Struct

capnpy turns each capnproto struct into a Python class. The API is inspired by namedtuples:

  • the fields of the struct are exposed as plain attributes
  • objects are immutable; it is not possible to change the value of a field once the object has been instantiated. If you need to change the value of a field, you can instantiate a new object, as you would do with namedtuples
  • objects can be made comparable and hashable by specifying the $Py.key annotation

Moreover, in case the type of a field is a pointer (e.g. Text, Data, structs and lists), capnpy generates two different accessors. For a field named foo:

  • has_foo(): return True if foo is not NULL, False otherwise
  • get_foo(): if has_foo() is True, it is equivalent to foo. Else, it returns the default value for that field

Note that in case of a struct field, the default value is a struct whose fields have all the default value, recursively:

@0xe62e66ea90a396da;
struct Point {
    x @0 :Int64;
    y @1 :Int64;
    name @2 :Text;
}

struct Rectangle {
    a @0 :Point;
    b @1 :Point;
}
>>> mod = capnpy.load_schema('example_struct')
>>> p = mod.Point()
>>> p
<Point: (x = 0, y = 0)>
>>> print(p.name)
None
>>> p.has_name()
False
>>> p.get_name()
''
>>> rect = mod.Rectangle()
>>> print(rect.a)
None
>>> print(rect.has_a())
False
>>> print(rect.get_a())
<Point: (x = 0, y = 0)>
>>> rect.get_a().get_name()
''

The rationale is that get_foo() and has_foo() are modeled after the semantics of the original C++ implementation of capnproto, while .foo is modeled after the Pythonic namedtuple API. In particular .foo returns None instead of the default value to avoid unpythonic and surprising cases such as Point(name=None).name == ''

Enum

capnproto enums are represented as subclasses of int, so that we can easily use both the numeric and the symbolic values:

@0x8eecd1de76ded4c4;
enum Color {
    red @0;
    green @1;
    blue @2;
    yellow @3;
}
>>> mod = capnpy.load_schema('example_enum')
>>> Color = mod.Color
>>> Color.green
<Color.green: 1>
>>> int(Color.green)
1
>>> str(Color.green)
'green'
>>> Color.green + 2
3
>>> Color(2)
<Color.blue: 2>
>>> Color.__members__
('red', 'green', 'blue', 'yellow')

Union

capnproto uses a special enum value, called tag, to identify the field which is currently set inside an union; capnpy follows this semantics by automatically creating an Enum whose members correspond to fields of the union.

@0x8ced518a09aa7ce3;
struct Shape {
  area @0 :Float64;

  union {
    circle @1 :Float64;      # radius
    square @2 :Float64;      # width
  }
}

struct Type {
  union {
    void @0 :Void;
    bool @1 :Void;
    int64 @2 :Void;
    float64 @3 :Void;
    text @4 :Void;
  }
}
>>> mod = capnpy.load_schema('example_union')
>>> Shape, Type = mod.Shape, mod.Type
>>> Shape.__tag__
<class 'example_union.Shape__tag__'>
>>> Shape.__tag__.__members__
('circle', 'square')
>>> Type.__tag__.__members__
('void', 'bool', 'int64', 'float64', 'text')

You can query which field is set by calling which(), or by calling one of the is_*() methods which are automatically generated:

>>> s = Shape(area=16, square=4)
>>> s.which()
<Shape__tag__.square: 1>
>>> s.__which__()
1
>>> s.is_circle()
False
>>> s.is_square()
True

The difference between which() and __which__() is that the former return an Enum value, while the latter a raw integer: on CPython, which() is approximately 4x slower, so you might consider to use the raw form in performance-critical parts of your code. On PyPy, the two forms have the very same performance.

Since capnpy objects are immutable, union fields must be set when instantiating the object. The first way is to call the default constructor and set the field as usual:

>>> s = Shape(area=3*3*math.pi, circle=3)
>>> s.is_circle()
True

If you try to specify two conflicting fields, you get an error:

>>> Shape(area=16, square=4, circle=42)
Traceback (most recent call last):
  ...
TypeError: got multiple values for the union tag: circle, square

The second way is to use one of the special new_*() alternate constructors:

>>> s = Shape.new_square(area=16, square=4)
>>> s.is_square()
True
>>> s = Shape.new_square(area=16, square=4, circle=42)
Traceback (most recent call last):
  ...
TypeError: new_square() got an unexpected keyword argument 'circle'

The alternate constructors are especially handy in case of Void union fields, because in that case you don’t need to specify the (void) value of the field:

>>> t = Type.new_int64()
>>> t.which()
<Type__tag__.int64: 2>
>>> t.is_int64()
True

Groups

@0x97a960ad8d4cf616;
struct Point {
    position :group {
        x @0 :Int64;
        y @1 :Int64;
    }
    color @2 :Text;
}

Group fields are initialized using a tuple, and accessed using the usual dot notation:

>>> mod = capnpy.load_schema('example_group')
>>> Point = mod.Point
>>> p = Point(position=(3, 4), color='red')
>>> p.position.x
3
>>> p.position.y
4

capnpy also generates a group constructor, which is a staticmethod named as the capitalized group name. It is useful because you can use keyword arguments and get the desired tuple in the right order:

>>> Point.Position(y=6, x=5)
(5, 6)
>>> p2 = Point(position=Point.Position(x=5, y=6), color='red')
>>> p2.position.x
5
>>> p2.position.y
6

By using the group constructor, you can also omit some parameters; in this case, they will get the default value, as usual:

>>> Point.Position(y=7)
(0, 7)

Note

Make sure to notice the difference between the lowercase Point.position which is a property used to read the field, and the capitalized Point.Position which is the group constructor:

>>> Point.position
<property object at ...>
>>> Point.Position
<function Position at ...>

Virtual groups

You can use the $Py.group annotation on a Void field to generate a virtual group, which fishes the data from normal “flat” fields.

@0x97a960ad8d4cf616;
using Py = import "/capnpy/annotate.capnp";

struct Point {
    x @0 :Int64;
    y @1 :Int64;
    color @2 :Text;
    position @3 :Void $Py.group("x, y") $Py.key("*");
}

This becomes particularly handy in conjuction with $Py.key (see Equality and hashing), because it allows to get an hashable/comparable subset of the fields without affecting other parts of the code which want to access the flat fields:

>>> mod = capnpy.load_schema('example_py_group')
>>> p = mod.Point(x=1, y=2, color='red')
>>> p.x
1
>>> p.position.x
1
>>> p.position == (1, 2)
True

Named unions

Named unions are a special case of groups.

@0xe1f94ddddf8858c4;
struct Person {
  name @0 :Text;
  job :union {
      unemployed @1 :Void;
      employer @2 :Text; # this is the company name
      selfEmployed @3 :Void;
  }
}

You can instantiate new objects as you would do with a normal group, by using the group constructor. If you want to specify a Void union field, you can use None:

>>> mod = capnpy.load_schema('example_named_union')
>>> Person = mod.Person
>>> p1 = Person(name='Alice', job=Person.Job(unemployed=None))
>>> p2 = Person(name='Bob', job=Person.Job(employer='Capnpy corporation'))

Reading named unions is the same as anonymous ones:

>>> p1.job.which()
<Person_job__tag__.unemployed: 0>
>>> p1.job.is_unemployed()
True
>>> p2.job.employer
'Capnpy corporation'

Note

The reason why you have to use the group constructor is that it automatically insert the special undefined value in the right positions:

>>> from capnpy.struct_ import undefined
>>> undefined
<undefined>
>>> Person.Job(unemployed=None)
(None, <undefined>, <undefined>)
>>> Person.Job(employer='Capnpy corporation')
(<undefined>, 'Capnpy corporation', <undefined>)

“Compact” structs

A struct object is said to be “compact” if:

  1. there is no gap between the data and pointers sections
  2. there is no gap between the children
  3. the pointers to the children are ordered
  4. the children are recursively compact

The compactness of a message depends on the implementation which generates it. The most natural way to generate Cap’n Proto messages is to write them in pre-order (i.e., you write first the root, then its children in order, recursively). If the messages are generated this way and without introducing gaps, it is automatically compact.

Messages created by capnpy are always compact.

You can check for compactness by calling the _is_compact method:

>>> mod = capnpy.load_schema('example_compact')
>>> p = mod.Point(1, 2)
>>> p._is_compact()
True

List items

Cap’n Proto lists are implemented in such a way that items are placed one next to the other, and the children of the items are placed at the end of the list body. This means that, if the items have children, surely there will be a gap.

Hence, as soon as you have a Cap’n Proto list whose items have pointers, the items are not compact, even if the list as a whole is.

>>> mod = capnpy.load_schema('example_compact')
>>> p0 = mod.Point(1, 2, name='p0')
>>> p1 = mod.Point(3, 4, name='p1')
>>> poly = mod.Polygon(points=[p0, p1])
>>> poly._is_compact()
True
>>> poly.points[0]._is_compact()
False

The compact() method

Cap’n Proto message can be arbitrarly large and occupy a big amount of memory; moreover, when you access a struct field or a list item, the resulting object keeps alive the whole message.

However, sometimes you are interested in keeping alive only a smaller part it: you can accomplish this by calling the compact() method, which creates a new, smaller message containing only the desired subset. Also, as the name suggests, the newly created message is guaranteed to be compact:

>>> mod = capnpy.load_schema('example_compact')
>>> poly = mod.Polygon([mod.Point(1, 2, 'p0'), mod.Point(3, 4, 'p1')])
>>> len(poly._seg.buf)
80
>>> p0 = poly.points[0]
>>> len(p0._seg.buf)   # p0 keeps the whole segment alive
80
>>> p0._is_compact()
False
>>> pnew = p0.compact()
>>> len(pnew._seg.buf) # pnew keeps only a subset alive
40
>>> pnew._is_compact()
True

Equality and hashing

By default, structs are not hashable and cannot be compared:

>>> p1 = example.Point(x=1, y=2)
>>> p2 = example.Point(x=1, y=2)
>>> p1 == p2
Traceback (most recent call last):
  ...
TypeError: Cannot hash or compare capnpy structs. Use the $Py.key annotation to enable it

By specifying the $Py.key annotation, you explicitly tell capnpy which fields to consider when doing equality testing and hashing:

@0xaff59c0b39ac4242;
using Py = import "/capnpy/annotate.capnp";

# the name will be ignored in comparisons, as it is NOT in the key
struct Point $Py.key("x, y") {
    x @0 :Int64;
    y @1 :Int64;
    name @2 :Text;
}
>>> mod = capnpy.load_schema('example_key')
>>> Point = mod.Point
>>> p1 = Point(1, 2, "p1")
>>> p2 = Point(1, 2, "p2")
>>> p3 = Point(3, 4, "p3")
>>>
>>> p1 == p2
True
>>> p1 == p3
False

You can also use them as dictionary keys:

>>> d = {}
>>> d[p1] = 'hello'
>>> d[p2]
'hello'

Tip

If you have many fields, you can use $Py.key("*") to include all of them in the comparison key: this is equivalent of explicitly listing all the fields which are present in the schema at the moment of compilation. In particular, be aware that if later get objects which come from a newer schema, the additional fields will not be considered in the comparisons.

Moreover, the structs are guaranteed to hash and compare equal to the corresponding tuples:

>>> p1 == (1, 2)
True
>>> p3 == (3, 4)
True
>>> d[(1, 2)]
'hello'

Rationale

Why are not structs comparable by defaults but you have to manually specify $Py.key? Couldn’t capnpy be smart enough to figure out by itself?

We choose to use $Py.key because it is not obvious what is the right thing to do in presence of schema evolution. For example, suppose you start with previous version of struct Point which contains only x and y:

struct OlderPoint {
    x @0 :Int64;
    y @1 :Int64;
}
>>> OlderPoint = mod.OlderPoint
>>> p1 = OlderPoint(1, 2) # there is no "name" yet

Then, you receive some other object created with a newer schema which contains an additional field, such as our Point. Since Point is an evolution of OlderPoint, it is perfectly legit to load it:

>>> p_with_name = Point(1, 2, 'this is my name')
>>> message_from_the_future = p_with_name.dumps()
>>> p2 = OlderPoint.loads(message_from_the_future)
>>> p2.x, p2.y
(1, 2)

Now, note that the underyling data contains the name, although we don’t have the name field (because we are using an older schema):

>>> hasattr(p2, 'name')
False
>>> 'this is my name' in p2._seg.buf
True

So, what should p1 == p2 return? We might choose to simply ignore the name and return True. Or choose to consider p1.name equal to the empty string, or to None, and thus return False. Or we could declare that two objects are equal when their canonical representation is the same, which introduces even more subtle consequences.

According to the Zen of Python:

Explicit is better than implicit.
In the face of ambiguity, refuse the temptation to guess.

Hence, we require you to explicity specify which fields to consider.

Extending capnpy structs

As described above, each capnproto struct is converted into a Python class. With capnpy you can easily add methods by using the __extend__ class decorator:

>>> import math
>>> import capnpy
>>> Point = example.Point
>>>
>>> @Point.__extend__
... class Point:
...     def distance(self):
...         return math.sqrt(self.x**2 + self.y**2)
...
>>>
>>> p = Point(x=3, y=4)
>>> p.distance()
5.0

Although it seems magical, __extend__ is much simpler than it looks: what it does is simply to copy the content of the new class body Point into the body of the automatically-generated example.Point; the result is that example.Point contains both the original fields and the new methods.

When loading a schema, e.g. example.capnp, capnpy also searches for a file named example_extended.py in the same directory. If it exists, the code is executed in the same namespace as the schema being loaded, meaning that it is the perfect place where to put the __extend__ code to be sure that it will be immediately available. For example, suppose to have the following example_extended.py in the same directory as example.capnp:

# example_extended.py
import math

# Note that the Point class is already available, as this code is executed
# inside the namespace of the module loaded from example.capnp
@Point.__extend__
class Point:
    def distance(self):
        return math.sqrt(self.x**2 + self.y**2)

Then, the distance method will be immediately available as soon as we load the schema:

>>> import capnpy
>>> example = capnpy.load_schema('example')
>>> p = example.Point(3, 4)
>>> print(p.distance())
5.0

Reflection API

Using the reflection API, it is possible to programmatically query information about a schema, for example what are the fields inside a struct.

The main entry point is the function capnpy.get_reflection_data(), which returns the metadata for a given module as an instance of ReflectionData.

>>> mod = capnpy.load_schema('example')
>>> reflection = capnpy.get_reflection_data(mod)

Under the hood, the capnp compiler produces a capnproto representation of the parsed schema, where most capnproto entities are represented by nodes. You can use get_node to get the capnproto node corresponding to a given Python-level entity:

>>> # get the node for the Point struct
>>> node = reflection.get_node(mod.Point)
>>> type(node)
<class 'capnpy.schema.Node__Struct'>
>>> node.displayName[-19:]
'example.capnp:Point'
>>> node.which()
<Node__tag__.struct: 1>
>>> node.is_struct()
True
>>> for f in node.struct.fields:
...     print(f)
...
<Field 'x': int64>
<Field 'y': int64>

Note

By default, reflection data is included into all compiled schemas. You can change the behavior by setting the option include_reflection_data to False.

Nodes vs Python entities

When compiling a schema capnpy generates Python entities from nodes: for example, Struct are compiled as Python classes, and fields as Python properties. Although closely related, they are not always equivalent: for example, Field.name is always camelCase, but the Python property might be called differently, depending on the compilation options.

For example, consider the following schema:

@0xe62e66ea90a396da;
struct Foo {
    myField @0 :Int64;
}

enum Color {
    lightRed @0;
    darkGreen @1;
}

To get the correct Python-level name, you can call reflection.field_name():

>>> mod = capnpy.load_schema('example_reflection')
>>> reflection = capnpy.get_reflection_data(mod)
>>> node = reflection.get_node(mod.Foo)
>>> f = node.get_struct_fields()[0]
>>> f
<Field 'myField': int64>
>>> reflection.field_name(f)
'my_field'

This works also for enums:

>>> node = reflection.get_node(mod.Color)
>>> node.is_enum()
True
>>> enumerants = node.get_enum_enumerants()
>>> enumerants[0].name
'lightRed'
>>> reflection.field_name(enumerants[0])
'light_red'

Inspecting annotations

The Reflection API provides methods to inspect capnproto annotations. Consider the following schema, in which we use custom annotations to map structs to database tables:

@0x801e5c7f340eaf8f;

annotation dbTable(struct) :Text;
annotation dbPrimaryKey(field) :Void;

struct Person $dbTable("Persons") {
    id @0 :UInt64 $dbPrimaryKey;
    firstName @1 :Text;
    lastName @2 :Text;
    school @3 :UInt64;
}

struct School $dbTable("Schools") {
    id @0 :UInt64 $dbPrimaryKey;
    name @1 :Text;
    city @2 :Text;
}

You can use has_annotation() and get_annotation() to query about them:

>>> mod = capnpy.load_schema('example_reflection_db')
>>> reflection = capnpy.get_reflection_data(mod)
>>> reflection.has_annotation(mod.Person, mod.dbTable)
True
>>> reflection.get_annotation(mod.Person, mod.dbTable)
'Persons'

The following shows a complete example of how to use annotations to create a simple dump of the DB structure. It is also worth noticing the usage of reflection.field_name() to convert from e.g. firstName to first_name:

>>> def print_table(node):
...     table = reflection.get_annotation(node, mod.dbTable)
...     print('DB Table:', table)
...     for f in node.get_struct_fields():
...         print('   ', reflection.field_name(f), end='')
...         if reflection.has_annotation(f, mod.dbPrimaryKey):
...             print(' PRIMARY KEY', end='')
...         print()
>>>
>>> for node in reflection.allnodes.values():
...     if reflection.has_annotation(node, mod.dbTable):
...         print_table(node)
...
DB Table: Persons
    id PRIMARY KEY
    first_name
    last_name
    school
DB Table: Schools
    id PRIMARY KEY
    name
    city

capnpy vs pycapnp

To be written