Before talking in-depth about data centricity, it makes sense to talk a bit about interoperability. For much more detail about interoperability and data modeling, I highly recommend this e-Learning video from

To start with the basics, you have to understand your goal for interoperability between systems. I will use a modern car as an example of a system, and the future goal of connected cars as an example of a system of systems.

A system:  Performs some function. In the case of a car the functionality this includes:  Drive from point A to point B, use sensors to detect vehicle maintenance issues (e.g., low tire pressure), use sensors to help a human avoid crashes, etc.

A system of systems:  Multiple individual systems coordinating, and providing a greater whole. In this example, this includes multiple connected cars communicating to spread information about the driving conditions ahead to prevent accidents.

When you are building a system of systems, you have to take into account several non-functional requirements that affect how your systems behave. These functional requirements include:

  • Interchangeability: A system can be replaced, and the overall functionality of the system-of-systems remains the same
  • Replaceability: A system can be replaced, but may not provide the same capabilities as the original. The functionality of the system-of-systems will not be the same.
  • Extensibility:  A new system can be added, providing new capabilities
  • Integratability: A combination of systems can be composed into a functioning whole.

In the connected-car world, this means:

  • Interchangeability:  All cars provide the same functionality to the system-of-systems, including, for example, information about the driving conditions ahead.
  • Replaceability:  Some cars may provide less information, or different information about the driving conditions ahead. However, this information can still be used to help form a picture about upcoming hazards.
  • Extensibility:  Newer vehicles will provide additional information about driving conditions that were not available in older models. Some cars can use this new information; others cannot, but it will not harm the overall system-of-systems. It will evolve over time as more new vehicles can process this new information.
  • Integratability: Vehicles communicate in a way that is well-defined enough that algorithms can be added to individual systems or the system-of-systems that provide emergent behavior; vehicles cooperating in ways that prevent accidents, help traffic flow, etc.

To meet these non-functional requirements, you need to look at the levels of interoperability within your system-of-systems. These are well-defined in modeling and simulation theory.

Levels of interoperability

Levels of interoperability

I will focus for now on levels 0- 3:

Level 0:  No interoperability. That is the current level of interoperability between cars.

Level 1: Technical Interoperability. This means that there is some common protocol defined so bits can be shared unambiguously. The connected cars meetup I wrote about in my last post was describing how they were sure that defining the hardware and protocols to be used to communicate between cars would be enough for interoperability. This might be enough, if every car defines exactly the same set of messages – though defining this set of messages is a challenge that will require standards. This is because level 1 allows replaceability of components, but not integratability or interoperability.

Level 2: Syntactic Interoperability. This means that there is a common protocol, and there is a common structure or common data format for exchanging information. At this level, a data model is defined, and each system must send and receive data described by that model. By defining the data that are being communicated, it is possible to integrate vehicles that have different capabilities – as long as they provide the data described in the data model, they can be integrated into the greater system, and algorithms can act on that model to help cars drive more safely. This level of interoperability supports interchangeability and integrateabillty.

Level 3: Semantic Interoperability. This means that the data has a meaning that is explicitly modeled, and shared between systems. When data has a common, well-defined meaning, interoperability can be achieved. At this level, algorithms can easily be developed for vehicles regardless of which manufacturers’ cars are communicating.

If car manufacturers support interoperability levels 2 or 3, they (or third parties) can develop algorithms that can go beyond looking at road conditions, or sharing common sensor data, but will be able to help traffic flow, help prevent accidents and back ups, and make driving more fun.

My concern with the connected-car world is that they are barely thinking about level 1 interoperability – and at that level, with no data model, and no shared meaning of the data, many of the benefits of having cars that communicate will be lost.

The Internet of Things vs. the Industrial Internet

A couple months ago I attended an Internet of Things meetup in Sunnyvale, where a representative of a major automotive manufacturer was talking about connected cars.  The discussion was really interesting, but there were a couple of things that struck me about his presentation.  The first one was that he conflated the concepts of the Internet of Things and the Industrial Internet.  The second one was that his company is at a very early stage of understanding the challenges of networking cars together from different manufacturers, with different capabilities, or even with different model years.

To address the first item:  The Internet of Things and the Industrial Internet share some core concepts.  They both represent a world where more and more devices are connected, and the information available from devices is exploding.  And to be completely fair to automotive manufacturers, a car can fit into either (or both) categories.  There are some major differences between these two concepts, though.

The Internet of Things refers to systems that are typically more consumer-oriented, such as thermostats that can be monitored remotely.  There may be a massive number of these devices, but they generally don’t need to be monitored or controlled with a resolution shorter than seconds.  In many cases they involve no control at all, and only monitoring.  If one of these devices goes down, it does not cause a massive catastrophe.  These systems do not necessarily have to be robust to disconnections.  In the automotive world, this might include a car’s infotainment system.

The Industrial Internet, on the other hand, refers to systems that are less consumer-oriented.  The systems that will be part of the Industrial Internet are manufacturing systems, medical systems, power systems, and similar.  These systems usually involve both monitoring and control, and they are monitored at a resolution of milliseconds.  If one of these systems goes down, it can potentially have expensive or catastrophic consequences.  Within a car, this would include any sensor networks that are scanning for pedestrians or road hazards.  In a connected car, this would include the communications between vehicles that have safety consequences.

There are going to be major communication challenges with connected cars that are just now being investigated.  It’s one thing to define common hardware that will be used to transmit data between cars, or even the low-level protocol that will be used.  It’s a very different thing to define what data is being transmitted between cars, to make sure that data has the same meaning across manufacturers, and to be able to provide new or more accurate data over time as new car models are developed with new sensors and capabilities.  And that isn’t even looking into the security concerns that come into play once massive numbers of vehicles are connected over a network.  A hacker could do some real damage if he or she could control networks of cars.

On the plus side, there are plenty of systems that are facing the same challenges, and some large organizations are working together to create solutions.  There are existing systems that already have to deal with the challenges of interoperability, future-proofing, security, robustness, and safety.  Some of the technologies that address these issues are being deployed in military and energy systems throughout the world.  One of the key concepts that they have embraced is that of “data centricity,” meaning that the data is the most important part of the system, and the interface between devices.

Next time: A little more about data centricity

The Encyclopédie

A couple years ago, while wandering through the Portobello Market in Notting Hill, a booth with old prints caught my eye.  A few of the prints seemed to be prints of scientific images, annotated in French.

Encyclopédie Image: Hydrostatique, Hydrodynamique et Hydraulique

Hydrostatique, Hydrodynamique et Hydraulique

I bought a page from that booth, having no idea what it was (and not yet knowing how bad it is to buy individual pages from rare books.)  I thought it was interesting because of its scientific content – it is a page titled “Hydrostatique, Hydrodynamique, et Hydraulique,” illustrating some properties of liquids and hydraulics, including an Archimedes screw used for pumping water.

A friend of mine who works with rare books pointed me to the fact that this is a page of the Encyclopédie, ou dictionnaire raisonné des sciences, des arts et des métiers, an 18th-century French encyclopedia.  I still didn’t realize how interesting this encyclopedia was until I read a book titled “Enlightening the World: Encyclopédie, The Book That Changed the Course of History,” by Phillip Blom.

This encyclopedia is an interesting – and surprisingly detailed – compendium of knowledge of science, art, and crafts gathered between 1750 and 1772.  Originally, the project was conceived to be a translation of an English encyclopedia, but the plan quickly changed under the editorship of Denis Diderot and Jean le Rond d’Alembert.  The Encyclopedia took contributions from a number of famous enlightenment thinkers, including Voltaire and Rousseau.

The philosophers who wrote the encyclopedia used this reference book as a way to subtly and sometimes not-so-subtly comment on their society.  One of their major targets was religion, and the religious hierarchy of 18th-century France.  This commentary was explicitly written in articles (usually in articles that didn’t specifically cover religion), and implicitly a part of the organization of the book itself.

The system of categorization that the book used was both new, and subversive to the religious thinking of the day.  First of all – and this seems so natural today that we forget that it was a new system – was listing all entries in alphabetical order.  This was not the first publication to be organized this way, but it was a recent innovation.  This allowed arts, science, and crafts to be figuratively at the same level as religious subjects.   Previous organization systems would list God as the highest level of knowledge, and then branches of human though or reason to fall under religious categories.  Alphabetizing the entires allowed all entries to be egalitarian.

Even more worrisome to the religious powers at the time was the incendiary introduction written by D’Alemebert (the second editor of the encyclopedia, but ultimately not as great a contributor as Diderot).  D’Alembert laid out the “Figurative System of Human Knowledge,” inspired by Francis Bacon.  At the top level this system includes Memory, Reason, and Imagination – nothing clearly religious.  Further down in the categorization, this categorization is even more explicit in challenging religion – it lays out theology under philosophy, rather than at the top level of human thought.

Various religious (and secular) figures tried to end the encyclopedia at different times, nearly halting the production of the Encyclopédie and by edict of Parliament declaring it dead in 1759.  However, the Encyclopedia became a large enough enterprise that it survived the attacks from religious and secular leaders, and was published in 28 volumes over twenty-five years.  Its influence shaped French society of the 18th century, and its philosophy was the direct precursor to the French Revolution.

If you’re more interested in the Encyclopédie, a few places to find more information:

Using tar on a VirtualBox shared drive

I set up my laptop using VirtualBox with several virtual machines.  The last time I set up a machine this way, I ended up with some VMs with large drives that I didn’t use, and I ran out of space on the host.  To avoid this on my new machine, I decided this time to keep my VMs as small as possible, and do all major work on shared drives.

So, I was running a build on the shared drive, when I hit this error:

Cannot utime: Operation not permitted

This error isn’t very helpful, but I was able to google around until I found out that this error can happen if you don’t have full ownership of the files you’re trying to tar.  The next step was to figure out why my user didn’t have full ownership of those files.

It seems that if you use the VirtualBox Guest Additions to share folders, you have no option to set any options for the mounted folder.  This means that all files are owned by root.  Most of the time this is not a problem, but tar was very unhappy with this.

You cannot edit the /etc/fstab file to mount the drive automatically, because the shared folder module is not loaded before the file is processed (see the VB forum link below). So, for the time being I have manually mounted the drive with options that give my user ownership over files:

sudo mount -t vboxsf -o uid=1000,gid=1000 <shared folder> <mount point>

According to the VirtualBox forums, it looks like you can add this line to the /etc/rc.local file to mount at startup.  As soon as I have to reboot I will try this.

3-D Printer

I’m going to get an opportunity to design something and build it using a 3-d printer in a few weeks.

Right now I’m contemplating what to build.  Something useful?  Something pretty?  Something fun?

The only restriction is that it will have to be something fairly small, because my budget is limited.

He Called Her… What??

The editor of Biology Online dot org asked a science blogger to write something for free, or for publicity (read: for free).  She politely turned him down.  He called her a whore.

The idea that somebody would call any woman a whore because she politely declined his offer of unpaid work gives me chills.  I can’t comprehend how the editor of Biology Online would write something so inappropriate.

The full story is described on Sean Carroll’s blog, where he has reproduced the original blog post that was posted in DNLee’s blog on the Scientific American site.  Apparently Scientific American removed the original post, claiming this was because the content was not about “discovering science.”


Girl Geek Dinner

I went to my first Girl Geek Dinner last week at the Intel Museum, and it was loads of fun.

I enjoyed the chance to play with some of the new laptops that use Intel chips.  They were showing off all the laptop-tablet crossovers, including models with sliding screens and the flip-screen.  These mostly felt too clunky to me, with the exception of the Sony.  The Sony Vaio was starting to be lightweight enough to feel like a tablet rather than a laptop.  This was also my first time playing with Windows 8, which I didn’t find immediately intuitive.

The speakers talked a lot about how they became leaders at Intel.  It was inspirational, and a few points have really stuck.

  • It is okay to make mistakes.  You do not need to be perfect all the time.
  • Stress is good – it is your body preparing for the next big thing.
  • The career ladder is not a ladder.  It is okay to make lateral moves, and to end up somewhere you never imagined.

And, I got to try a locked down Google Glass.  As my friend said:  “True nerd happiness.”

Google Glass

Using shapelib to Read Maps

I’ve just had my first fun in drawing maps based on data that is available from various public-domain government sources.  Most of the geography of the US (and the UK) is available in a shapefile format.  For example, you can download the boundaries of all the states of the US.

There are lots of tools to read shapefiles, depending on which language you are writing in – some are very simple, such as the libraries available for JavaScript.  My code is written in C++ so I chose the shapelib library, as it is the lightest-weight way to open them.

Opening Shape Files

The call to open a shape file is:

SHPOpen (filePath, permissions)

The first thing to note is that when you open a shapefile you do not specify the file you want to open, but instead you specify the directory that contains the .shp file, .prj file, and other supporting files.  In other words, if you want to open a shapefile called ne_50m_land.shp, you specify the enclosing directory as the path, such as:

SHPOpen("/path/to/directory/ne_50m_land", permissions);

The second thing to note is that the shape library is picky about the format of the permission strings.  If you want to open with read-write access, you must pass in “r+b”.  It does not accept “rb+” which can be an acceptable parameter to fopen().

 API Notes and Comments

After you open the shape file, the next thing you do is call SHPGetInfo() to fill in the information about your shape.

void SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType,
                 double * padfMinBound, double * padfMaxBound );

The parameters are documented here.  One key point that it is easy to miss if you are not reading the documentation carefully is that padfMinBound and padfMaxBound are both pointers to four-entry arrays that are filled in with the minimum and maximum bounding values in each of four dimensions — x, y, z, and m.

Your next step is to check what type of shape this is (SHPT_POINT, SHPT_ARC, SHPT_POLYGON, or SHPT_MULTIPOINT), and then iterate over the number of entities that was returned in the pnEntities parameter.  For each entity, you call

SHPObject *SHPReadObject( SHPHandle hSHP, int iShape );

This returns a SHPObject that contains the geometry for some shape within the map.

This geometry may represent more than one part, such as multiple polygons representing individual states or counties.  The nParts field indicates the total number of parts in the shape.  The vertices for all the parts are stored in single arrays for X values, Y values, Z values, and M values called padfX, padfY, padfZ, and padfM.  So, you have to determine which of the vertices belong to each shape by using the panPartStart array that lists the starting point for each shape.

Here is some example code that shows how to use nParts and panPartStart[] to access vertices for each individual part of the shape.

// Iterate over all of the shape objects in the shape file
for (unsigned int i = 0; i < shapeObjects.size(); i++) {

    // Iterate over the parts of the shape
    for (int j = 0; j < shapeObjects[i]->nParts; j++) {

        // Maintain multiple lists of points.  Each list contains
        // all of the points required for a single polygon.
        list<Point *> *points = new list<Point *>();
        int nVerticesPerPart = 0;

        // Determine which part (unique polygon) of the shape
        // these points belong to.
        if (j == shapeObjects[i]->nParts - 1) {
            int startOfPart = shapeObjects[i]->panPartStart[j];
            nVerticesPerPart = shapeObjects[i]->nVertices - 
        } else {
            nVerticesPerPart = shapeObjects[i]->panPartStart[j + 1] - 

        // Add the points from the current shape part to a list.
        // We will create multiple lists of points - one for each shape part
        for (int k = shapeObjects[i]->panPartStart[j]; k < 
                     shapeObjects[i]->panPartStart[j] + nVerticesPerPart; k++) {

            Point *newPoint = new Point(shapeObjects[i]->padfX[k], 

            // Add this point to the end of a list of points

    // Add this list of points to a list of point lists

Projections:  Mercator, Lat/Long, etc.

Aside from examining the maximum and minimum values within the shapefile (and the supporting documentation downloaded with an individual shapefile), I did not find any explicit information about what coordinate/projection system the shapefile is using.

Additional Information

Details on the shapefile specification can be found here:


“We Have Male Engineers, Too… “

My mom is a software engineer, I’m a software engineer, my sister is studying to be a software engineer.  The first video game I ever played was written for me by my mom and dad.  I grew up with computers, and I knew from a very young age I was going to be a software engineer.

But it was still a pretty telling moment when one of my fellow women-in-engineering jokingly told our customer that “we have male engineers, too.”  It’s hilarious, but it really shouldn’t be.

I grew up in Silicon Valley, which means that I had a number of advantages that other people didn’t have – such as having a computer and Internet access long before most people.  Now those technologies are ubiquitous, with tablets and phones bringing the technology into every home, but I don’t see that making a big difference in who becomes engineers.  Increasingly those technologies are created to make people the consumers of devices, instead of tinkerers.  Technology has become something to own, not something to understand.

The greater advantage that I had growing up here is the culture.  Before we had the culture of startups, IPOs, and get-rich-quick, we had the culture of smart people solving hard problems.  I grew up knowing that engineering can be interesting and fulfilling – and not just the domain of some group of socially-awkward men.  Many places do not have this culture – and without it, you don’t have women engineers, or engineers in general.

I had a struggle with culture shock when I moved to London.  Just one example is when a man flat out exited the conversation when I mentioned the words “space robots.”  It wasn’t because he was afraid of the concept, or really had anything much to say about it – he was just suddenly categorizing me as a nerd, and uninteresting.  In Silicon Valley, this elicits the exact opposite response, regardless of whether people are thinking of the Mars Rover or Skynet.

In the UK, a lot of engineering is categorized under the umbrella of IT, and I think this says something about the culture.  There’s a certain smugness that City types have when thinking about the nerds who do that infrastructure stuff.  It’s as though they control the big picture while their IT staff just provide some help to allow them to achieve their goals (i.e., get rich).

Some places I’ve worked are even more surprising – either in how they regard technology, or how they regard women in technology.   Some places were better than I expected, some worse.  But I know that we can do better, even here in Silicon Valley.  Women need to know that they can be engineers, that they can be good engineers, and that they can enjoy it.  It’s one way to shape the world that we are building for ourselves, instead of just consuming what others produce.