Chapter 15: Mapping an XML Feed

The vaccination campaign needed transport. The regional bus company was happy to help, and sent its timetable feed: Transport.xml. “It’s what our ticket machines read,” the dispatcher said. “It’s XML, of course.”

Dr. Portbridge had just learned to map the board’s JSON. The tags looked different, but the question was the same one: where does one record start, and where does each value sit inside it?

The problem

Chapter 14 mapped a JSON document by pointing at its values with JSONPath. XML works the same way, with a different path language: XPath. A mapping still needs to know what counts as one record and where each value sits inside it; only the way of writing those paths changes.

The file

Here is the bus company’s feed:

<transport>
  <bus id="25">
    <route>
      <stop id="645">Central</stop>
    </route>
  </bus>
  <bus id="26">
    <route>
      <stop id="700">Harbour</stop>
    </route>
  </bus>
</transport>

Two buses, each with a route and one stop. A stop carries its number in an id attribute and its name as text. We want a bus and a stop out of each bus element, linked together.

The whole mapping


prefix <http://vocab.org/transit/terms/> as transit
prefix <http://www.w3.org/2000/01/rdf-schema#> as rdfs

mapping from <package://Transport.xml> as XPath
  for each "/transport/bus"
  map:
    "@id" as ?id
    "route/stop/@id" as ?stop_id
    "route/stop" as ?stop_label
  to:
    fact <http://trans.example.com/bus/{?id}> a transit:Bus
      transit:stop <http://trans.example.com/stop/{?stop_id}>
    fact <http://trans.example.com/stop/{?stop_id}> a transit:Stop
      transit:code {?stop_id as int}
      rdfs:label {?stop_label}

It reads top to bottom in the same four steps as the JSON mapping.

1. The source

mapping from <package://Transport.xml> as XPath names the file, relative to the package root, and picks the XML reader. Values will be addressed with XPath strings: / steps into a child element, @ reads an attribute.

2. `for each`: what is a row?

for each "/transport/bus" says: every bus element under the root transport element is one record. The feed has two, so the to: block runs twice.

Leave the line out and the whole document is a single record, exactly as with JSON.

3. `map:`: where the values are

Each line binds a path to a variable. The paths are relative to the current bus:

"@id" reads the bus’s own id attribute;
"route/stop/@id" steps into its route, then its stop, and reads the stop’s id;
"route/stop" stops at the stop element itself. An element that holds only text binds that text, so this gives the stop’s name.

The same stop element therefore gives two values: its number and its name.

An XPath string literal uses single quotes, as in "stop[@kind='express']", so that it does not end the Dolfin string around it. Square brackets pick among several matches: "route/stop[2]" is the second stop of the route.

4. `to:`: what to build

There are two fact templates, and both run for every bus:

the first builds the bus, named after ?id, with a link to its stop;
the second builds the stop, named after ?stop_id, with a code and a label.

The link works because both templates build the same IRI, <http://trans.example.com/stop/{?stop_id}>, from the same variable.

XML values bind as plain text, like JSON values and CSV cells. The {?stop_id as int} cast, written inside the braces, turns the text "645" into an integer literal.

What comes out

Building the package expands the mapping before Turtle is generated, so each bus becomes two subjects:

<http://trans.example.com/bus/25> rdf:type owl:NamedIndividual
  , transit:Bus
  ; rdfs:isDefinedBy <http://example.org/transport/transport>
  .
<http://trans.example.com/bus/25> transit:stop <http://trans.example.com/stop/645> .

<http://trans.example.com/stop/645> rdf:type owl:NamedIndividual
  , transit:Stop
  ; rdfs:isDefinedBy <http://example.org/transport/transport>
  .
<http://trans.example.com/stop/645> transit:code "645"^^xsd:integer .
<http://trans.example.com/stop/645> rdfs:label "Central" .

<http://trans.example.com/bus/26> rdf:type owl:NamedIndividual
  , transit:Bus
  ; rdfs:isDefinedBy <http://example.org/transport/transport>
  .
<http://trans.example.com/bus/26> transit:stop <http://trans.example.com/stop/700> .

<http://trans.example.com/stop/700> rdf:type owl:NamedIndividual
  , transit:Stop
  ; rdfs:isDefinedBy <http://example.org/transport/transport>
  .
<http://trans.example.com/stop/700> transit:code "700"^^xsd:integer .
<http://trans.example.com/stop/700> rdfs:label "Harbour" .

Each subject is typed by its template (transit:Bus, transit:Stop) and carries its mapped properties: one for the bus, two for its stop.

When a path does not match

The rules are the ones you already know from JSON:

a path that matches nothing gives an empty value, and if that value builds the subject IRI the whole bus is skipped with a diagnostic;
a path that hits an element with child elements ("route") skips the bus: a whole subtree is never turned into a text literal;
a path that hits several nodes binds the first one and warns. A route with two stops would do that with "route/stop/@id"; to get one record per stop, iterate over the stops instead with for each "/transport/bus/route/stop".

Real feeds often put their elements in an XML namespace. Declare it with an ordinary prefix line and use that prefix in the path ("/t:transport/t:bus"). The reader only understands a fixed subset of XPath, and anything outside it is an error rather than a silent empty match. The syntax reference lists every rule, including namespaces and the supported XPath subset.

The two buses, their stops and the links between them were in the system in seconds. When the company added a third line the following week, the dispatcher sent a new feed. Dr. Portbridge replaced the file and rebuilt. The mapping did not change.

Keyboard shortcuts

Dolfin Documentation