Escher Looking Glass by whendt

previous parts can be found here :

Is the digital world really like spaghetti ?

The fight against technical debt is one of the CTO/CIO major challenge. What's funny with the technical debt is that we often associate it to spaghetti: a clear picture that says, if you try clean a small part, you'll probably have to clean everything. But there is bias coming from this spaghetti idea we have of the IT mess:

  • We are over-estimating links and under-estimating contextual knowledge on linked components;
  • We are considering the nodes like black-box, yet these black-box can really be complex, besides the fact that it keeps changing over time.


The spaghetti nightmare


The Cabling Technical Debt: Awful Yellow Spaghetti.

What we expect to see on any data-center: clean and labeled cabling.
Note : The label on the cable gives you an entry to understand the context of the cable.

Most of the visualization tools we saw are graph (nodes and links) based. Seen in a large scale, these visualizations will also look like spaghetti, and it is funny to see how we are reducing our visualization capabilities to what we know (or what we believe to be true paradigm).

The digital world is much more like connected broccoli

Reducing IT to spaghetti was probably a good idea at the time we were racking and pulling network cables everywhere across the data centers, but these times are over. This is virtualization and Software As Everything time and consequently IT, is much much more dense than any spaghetti.

For those who are not experts, the artistic draw which can probably give a better idea of how this digital world looks like, are some fractal draws. Indeed, these fractal draws have something common with the digital world: There are component repetitions everywhere!

Box city fractal



When mapping the digital world we're trying to map something we can not see. We - as operational expert - can only imagine how the runtime system looks like according to some architecture diagrams and do the reconciliation with some technical operations on the runtime. The technical operations done on the runtime will define our expertise and our silo in the organization.

Some organizations may have a LOT of silos/experts and most of the time there is a lot of work to improve the communication between these silos/experts. This is one of DevOps target. And clearly one of the major weakness of current DevOps tools is visualization and knowledge management.


Fractals in Ghost In The Shell


Information System visualization in Ghost In The Shell. This guy browses the running system
like I'm able to walk in some unknown city with google maps.


And finally, Ghost In The Shell visualization tools seem to be much more complete, realistic and usable
than any other product on our present market. Note the repetitive graphic patterns which could be a big fractal base.


When looking back at the software layer and the developers work, there is something fascinating when you realize that whatever the language and the complexity, they are all following the same basic patterns and models. These simple patterns and models allow you to build almost everything ! The reason why I personally prefer to compare the digital world to connected broccoli instead of spaghetti...


Brocolis is a fractal too ;)


Improving the digital world mapping with fractals and graphs

Some refresh about fractals

Fractal word has been introduced by Benoit Mandelbrot in 1975, but is the result of four centuries of mathematical research.


Mandelbrot definition of fractals


Fractal is "a rough or fragmented geometric shape that can be split into parts, each of which is (at least approximately) a reduced-size copy of the whole"


Fractals are particularly interesting when it comes to setup efficient visualization algorithms to build rough objects. It begins with an initiator which can be a really simple euclidian object, and a generator which is a simple rule you can iterate infinitely from the initiator.

Koch snowflake


Koch snowflake is a famous fractal following self-similar pattern.
It illustrate perfectly how we can build easily a really complex object
by incrementing simple geometric operations.

Koch generator and initiator.
Notice how the final result can change depending on the fractal generator.
Generating Fractals Based on Spatial Organizations

Random Koch


Randomization of Koch algorithm lead to some realistic natural landscape

Therefore fractals are widely used in the computer game or movie industry helping generating realistic natural landscapes.

Fractal Mountain


Fractal geometry help to picture mountain with few iterations and simple algorithm.
Hence the reason why fractals have been used really early to produce
genesis sequence landscape in Star Trek.

Improve the digital world visualization means improve the digital world fractal initiator

It has recently been demonstrated that complex networks are self-similar. That means they follow some power law, and that the fractal geometry tools can be applied to the study of these networks.

When speaking about the world-wide web, we're generally describing a big graph with nodes as routers and links between. But this modelization is also inefficient, as it limits the routers, just like a black-box, and, consequently, it is not helpful for the expert who needs to get the running structures of these routers. Basically, there is another graph inside the router, and it is part of the global world wide graph.

The ability to look inside the digital components becomes more and more important as long as we're pushing more and more business on these components, and as long as we're densifying the digital architecture with new technologies like VMs or containers.

Considering the graph objects (IE: nodes and links) as the world wide network's fractal initiator, it would be legitimate to improve this initiator, in order to provide a new and complete way to visualize the complex runtime we may setup in many business. This will be discussed in the next article.


(previous part can be found here : A trip through the Ariane project - Part 1 : Make fun and profits from I.T burst)

Visualization is a key factor for any software sales. One of the most definitively sexy visualization tools to show to any CIO/CTO is mapping. This is the reason why you will see lots of products giving you a taste of IT mapping, even if they don't have same purposes and work differently. Let's go on a little tour of the market...

The ITSM tools

Basically, they play a referential role with some CMDB, plus some mapping visualization and links to ticket tracking for purposes of following changes on each object in the referential. Most of them provide some automated discovery on the physical assets in your data centers, and then run some commands on the identified assets to look at the services running on these assets and their relationships.

These tools are able to display some IT map but regarding the granularity (assets and services), given that they are targeting the managers or high level architects. Actually, this granularity doesn't fit the OPS and DEV teams who are looking for services and assets network contexts, and internal details to understand or fix a problem. Unfortunately, the devil is in the details...


BMC Discovery





The monitoring and APM tools

Performance monitoring and alerting form the basis for monitoring and other APM functionality, but these tools offer some maps as well. There are many monitoring tools, and all of them have a specific target: System OPS, Database OPS, Middle-ware OPS, Application OPS .... This means you will have as many tools as teams and silos you find in your company. Then, to get the complete map - or at least a complete part of your global map - you'll need data / expertise reconciliation, which could take time...

System monitoring with Nagios


Probably one of the oldest system monitoring solution on the market, Nagios offers several visualization tools.
Among them this one giving you the layer 3 map of your network and the devices on it.

Network monitoring with Netbrain


Afficher l'image d'origine

Netbrain can help your discover the network (layer 3 or layer 2) path between two points (and more)

Application monitoring with ITRS Geneos


ITRS Geneos help you build dashboard aggregating several monitoring data coming from your distributed system.
This is a pretty map indeed with contextual data ... Just one problem : must be done/updated manually.

Application Performance Monitoring with AppDynamic


AppDynamic is a famous APM which can show you the world from the application perspective (Layer 5).

Some other inspiring maps

Netflix microservices visualisation tool


Netflix is working hard to master its microservices architecture. Here is another visualization example.
Extract from "A Microscope on Microservices" :
"This internal utility, Slalom, allows a given service to understand upstream and
downstream dependencies,their contribution on service demand,
and the general health of said requests.

Simianviz / Spigo



@adrianco, the father of Netflix Microservices architecture, is currently working on a
simulation tool to visualize different architectures (NetflixOSS, Lamp ...).

Kubernetes Visualization


Another specialized graph for Kubernetes users...
Note the contains links which could leads to a bigraph visualization.


Is there anything missing ?

As has already been said, these specialized maps are the result of extreme OPS expertise silozation. But wait: Aren't we forgetting some population here? DEV ? Aah yeah! Is there any kind of map for DEV dudes? Like something helping visualizing threads or actors? Not really. DEV are using some tools like Kcachegrind or other profiler to browse the stack and see where time is consumed.

Kcachegrind in action


Afficher l'image d'origine

Used on DEV environment only, profiler like Kcachegrind are helpul for developers.

Unfortunately these tools consume a LOT of CPU memory and therefore are never used in production, and some times they consume so much of it that you're unable to reproduce a problem using them... And finally, they are not showing the threads/actors topology, but aggregated statistics on the run stack and calls order tree .

Akka actors in action


Pub Sub

This is how @hicolour specified a scalable load balancer with Akka. There is currently no tool able to represent this granularity.
However, all this definition of actors are the keys to understand how this dynamic system works

But many problems are coming from un-mapped components like processes, threads or actors. Finally, these components are the basis for the application's architecture. The ability to map such components brings us to a new area where OPS can communicate with DEV much more easily and so solve problems more efficiently.

Next articles :

A trip through the Ariane project - Part 3 : make fun and profits from fractal theory


First of all, Ariane is a Free Open Source Software (F.O.S.S) Project. But don't believe that because it's free means we are being philanthropic only.


Actually, since the times of Alan Turing, the evolution of information technologies has led us to believe that information technologies count function is at least regulated by some kind of power law ((o) xk with some k > 1 (1)) . Indeed, this evolution is defined by:

  • layer effect : much of the technology on some layer is meant to be completed by another one in layer above
  • high competition : for the same functionality you're able to find a lot of competitors with their pros and cons.
  • open source : open source  policy is a boost to innovation, which empowers anyone to add new technology or improve existing


Example of power law : the square cube law



Graphs of surface area, V against area, A of the Platonic solids and a sphere,
showing that the surface-volume-to-area ratio increases with increasing area such that V = A k; k=3/2 > 1.

In our bet, we expect the growth of information technologies to follow a power law with k by >1.
This has a significant impact on our problem analyses, our solution design and target...


We will not rigorously prove this assumption in this set of articles set, but for those who are interested on such demonstration, a good start would be to have look at trees of dependencies, which furiously look like the fractal tree.


The dependency graph looks like fractal tree



In the long term, you had better get some good trust relationships with all this new robot and AI coming play with you. The best way to do this is to understand how all these stuff works, and learn to play with it. Knowledge is power.

This is where the map concept comes from. This is really an old concept helping humans to learn faster, and this millennial field excels today with some applications like Google Maps linked to the Google Search engine: You can find any restaurants or bars and compute your path in any city you have never visited before...


Oldest known map


Imago Mundi Babylonian map, the oldest known world map, 6th century BCE Babylonia.



What about mapping on the IT/Digital world ? There are many attempts today on many different tools, but lots them are professional/private tools, and all provide good views which the users deserve, but are limited to the view the users deserve only. This is one of the problems today if you want to learn fast: You need to learn fast the technologies you work with and also learn fast the other tools helping you to manage them...

If you want to provide some great digital world mapping application like the one brought to you by Google maps for the physical world, you need to bridge the gap between:

  • the current IT mapping state of art : Specialized modernization where you need to get your data from lots of tools, expertise and meetings to build a complete map. 

  • what you think it should be : A complete map in one application which can help you understand the unknown world. 


You think that's crazy ? But let just remember our initial bet :


Initial bet


Information technologies count growth is regulated by some kind of power law ((o) xk with some k > 1)


Therefore looking to :

  • this IT count growth law provides you amazing choices to fit your needs
  • the business pressure to push new applications - and so new problems and so ... new technologies ...
  • and the average time needed to decommission a single one technology for rationalization purposes

Then you'll understand why even the best CTO/CIOs are struggling to rationalize their IT in the long term, and so the growing complexity and cost of their system.


This is where Ariane should be seen as a unique project: It pushes an ubiquitous data model for our digital world maps. From our point of view, pushing an ubiquitous mapping model to generate a unified map of your information system is really not a crazy idea but definitively the solution needed to master this complexity growth, which will obviously go with your business growth.

And as we're expecting a power law effect on the Ariane project life, we also chose to set up an open source and ultra modular project: a single company can NOT map all the digital world. But if you take a good look, you'll see every difficulty as an opportunity. By providing Ariane as an F.O.S.S project, we want:

  • To bring freedom of use for anyone to map their own and unique IT.

  • To setup an open eco-system to make (and share) fun and profit from this beautiful IT power law.

  • To reduce misunderstanding between IT professionals/ users by helping sharing knowledge across teams/ organizations.

  • To allow people to understand and control their digital world quickly.

Reading the following set of articles will give you some input on understanding how Ariane can help us to build a dynamic mapping service for our digital world. The next part is coming soon...

Next articles :

A trip through the Ariane project - Part 2 : a quick tour of I.T. visualization tools

A trip through the Ariane project - Part 3 : make fun and profits from fractal theory



Today, there is an abundance of information technologies (I.T) components that need to be mapped so that everyone can have access to full information system (I.S) mapping.

This paper presents Ariane which brings you an ubiquitous digital data pattern, allowing you to map any piece of your digital runtime in real time. Ariane is a framework that yields a highly modular system and API (Java or REST) to map any component you may have in your I.S. Once your stack is mapped, Ariane will help you to navigate the map from one point to another.

Finally Ariane can assist you to setup a new kind of I.T knowledge system with automated data miners getting the data from your living I.S and pushing them to a unified map which can be the base for a lot of new and cool analysis systems and UX features.




Information technologies (I.T) is a key enterprise factor today to acquire new clients, retain them and expand the business. Therefore the I.T engineers have developed significant amount of technologies and methodologies to acknowledge the business requests as quick as possible.  

Today, business solution usually means a lot of backend applications communicating with each others to perform the necessary computing which will make the final user contented. This application stack may also depends on a lot of  middleware, systems, networks and storages solutions. All these layers requires specific expertises and consequently the common pattern observed in the enterprise world is siloization (see [1] and consider I.T the same way as content).

As a result of this I.T siloization, deliveries process results in becoming more and more complex and requires a lot of coordination between the teams. This finally was bad, rather catastrophic for the time to market and so for the business (and probably business still suffers of this on many organisations). The primary actors concerned by this slow time to market was developers who were the first changing their work methodologies to Agile/XP methodologies and so speed up the releases.

But delivery was still slow as operational actors were focused on the stability of the production not speed. So a new trend emerges from this muddle to accelerate the deployment : the DevOps trend. DevOps pushed new automation systems and frameworks to the I.T (see [2],[3],[4],[5]) helping OPS improving the delivery reliability and speed. So finally OPS can be synced with the DEV as far as delivery speed is concerned, essentially helping to reduce the remoteness and the stress between these two actors. 

Automation is the new big term in any organization today, a lot of money and time being spent on it. But automation doesn’t act as a panacea for all the existent problems and particularly I.T expertise silos, which still prevails across the informations system. These silos still have a very bad effect on knowledge sharing and outage resolutions, which therefore can have a really bad effect on a company’s credibility ([6]). And finally I.T automation introduces a new problem, that being : the high frequency changes on information system, resulting in an impossibility to maintain I.S. Manual Human readable diagrams which are usually the first gate to help developers and operators understand the environment quickly, needs to be maintained and improved ([7]).

So now DevOps aficionados argues DevOps is not only about automation and accelerating deliveries, but is also about breaking down I.T silos and knowledge sharing ([8]) and this will be will be seen as the new big drift coming. But we also need to automate the knowledge base the same way we're automating the I.S delivery because as Human we need to keep the control over automated systems ([9]).

This is the primarily the mammoth target of the Ariane Project ([10]) which is a new free open source framework that helps users in writing mapping plugins to automate real time diagram generation of technologies they use every day. 




Ariane Architecture

In order to provide an extensible framework, Ariane is built on Eclipse Virgo ([11]) which implements OSGi standard ([12]). Such enactment helps us so that we can define a really fine grained modular system which will help us to have a maintainable system over time and clear separations of concern implementation through OSGi micro-services in a simple monolith to deploy ([13]). 

Development, in a way, can be related to cooking, and as in good cooking we need good ingredients, good development needs good libraries
. Therefore Ariane framework integrates additional component to Eclipse Virgo like Infinispan ([14]), Hibernate ([15]), Neo4J ([16]), Akka ([17]), Scala([18]), Groovy([19]), PrimeFaces([20])... These third party libraries aid in building a modern and beautiful web application.

Finally the Ariane framework provides features which are designed to be extended through Ariane plugins, primarily helping the end user to adapt the Ariane framework with its own information system environment. 

A. Security

The Ariane Identification Management component ([21]) is providing security service to Ariane system through Resources/Roles-Based Access Control (RRBAC) pattern ([22]). The resources, permissions and roles are stored into a basic SQL database (MariaDB ([23])) and so can be extended for any new plugins with simple SQL insert.

Currently, Users and Groups are also stored on the SQL database, thanks to Apache Shiro ([24]) we can easily provide drivers for other authentication realms through LDAP or ActiveDirectory repositories.

B. Portal component

The Ariane Portal component ([25]) is providing extendable web services. It handles the Ariane web application context and define OSGi micro-services to expand this web application through the following registries as under : 

  • the main menu items registry
  • the JSF bean controller registry
  • the REST endpoints registry (RESTEasy ([26])
  • the JSF ([27]) views registry 

Other Ariane components or plugins will just need to register their own menu entity, bean, views ... to extend the application UI. It also gives you a convenient and simple JavaServer Faces (JSF) verbs to request the Ariane IDM RBAC database in your views. 

C. Directories component

The Ariane Directories component provides ([28]) Ariane directories Java Persistence API (JPA - [29]) service to access the Ariane directories database. Ariane Directories entities are representation of your IT component definitions (like datacenters, networks, OS instances) and your organisation (applications, teams, environments).

The Ariane directories JPA service offers a convenient way to extend the entities definition in the running entity manager thanks to hibernate and OSGi (custom feature - [30]). So you can append other I.T component definitions like for example, any middleware containers you can have in your I.T to the Ariane directories. 

D. Mapping component

The Ariane Mapping component ([31]) provides Ariane mapping services relaying on Tinkerpop Blueprints ([32]), Neo4J and our ubiquitous I.T mapping model. You can inject data into the map, thanks to the mapping interface (currently memory call only) and provided REST endpoints.

It also dispenses our mapping DSL to request the map which has to be rendered in your browser throught the Ariane Mapping graph render (called Taitale - Daedalus in etruscan language) implemented with RaphaelJS ([33]).

E. Injectors component

To help you build Ariane plugin which are sniffing your information system and inserting the topology of your I.S into Ariane Mapping database, Ariane Injectors component ([34]) provides following tooling:

  • Cache factory to create on demand Infinispan cache for you components
  • Components commodities which gives you a pattern to write the I.S component data miners
  • Gears you can administrate through the Ariane UI, which are handling runtime behavior of your Ariane plugin.

There are two kind of injectors :

  • Internal injectors that are running into the Ariane Virgo containers and are calling the Ariane Java API (memory call yet but we will provide some distribution through Akka framework soon).
  • External injectors that are running outside the Ariane Virgo containers and are calling the Ariane network API (REST and ESB through RabbitMQ([35])). They can then be written in the language of your comfort.




Ariane framework’s key component is its mapping component which is employed to create unified map for your information system components. It contains of micro services, that empowers us to create a unified map. Ariane mapping services rely on Neo4J which is a scalable graph database, as against the classical SQL database, thus bypassing copious amounts of table joins. 

On top of classical GraphDB model (vertex and edges), Ariane provides an information system mapping model with advantages to be a simple and ubiquitous model. This model indeed has been inspired by several inter I.S communication protocols like AMQP ([36]) and it wont be an exaggeration to point that with not more than seven objects you are able to describe the most essential topology of your I.S coming from your I.S layers data (from low network technologies to highest applications technologies ([37])).

The Ariane mapping abstract domain model objects are as listed and described below :

A. Cluster

Cluster is set of containers with unique ID which may be anything, an active/active or active/passive cluster.
Cluster may know it’s current main container in realtime.

B. Container

Container is an administrable entity of your information system on which user can connect and authenticate. Container contains at least one administration gate which is the administration interface access. It contains one or multiple nodes which are logical entities dedicated for a specific tasks with unique ID. Container should be able to detect it’s own consitent state,  localization in real time (datacenter, network ....) and organisation context (developers team, support team ...)

C. Node

Node is a logical entity devoted to specific tasks like communication, computing, storage... It consists of a set of nodes and/or endpoints which is node address URL.
Node contains all necessary meta data for it’s description.

D. Gate

Gate is basically a type of node which is always a child of container but it assuredly cannot be a child of node. Gate defines services with at least one communication endpoint for remote calls. Gate must contain all necessary metadata defining the service.

E.  Endpoint

Endpoint is a node address URL. This URL is used for node to node communication (and hence, container to container). It is aware of the network stack on which it is working (for example : the network card).

A link is defined by two endpoints (source and destination). Link encloses its transport definition. In case of a multicast transport, link will have just the source endpoint alone.

G. Transport

Transport is a communication protocol used by the link which can be either unidirectional or bidirectional.




The Ariane Mapping Domain Specific Language (AMDSL - [38]) is developed with Scala to provide and ensure a fast and simple map query language, essentially based on the Ariane mapping model. User just needs to define the starting point(s), the target point(s) and the path (optional). The Mapping DSL grammar is pretty facile. It is a mixture of JSON, SQL and a pinch of Neo4J cypher (to define the path).

Basically it looks like :


Points are defined through SQL like syntax and the mapping object models. User can play with container, nodes, endpoints, repository to characterize the points they wish for their map. Path definition is not mandatory but recommended on some complex graph path traversal paths. This is because Ariane, by default, will check with all potential paths between/through the points you defined, which could consume some computing time you can reduce.




By using AMDSL, user can define a precise on demand map request which is then processed by the mapping back engine, which will return the list of objects found in the requested path through Ariane Mapping REST service. 

The final result is then generated by the Ariane mapping graph render (known as Taitale) which compute the positioning of the returned map objects and then displays them as SVG objects. All these objects and the links between them compose the result map. Once the map is loaded, user can steer through it, observe the map objects, properties and connect to the containers they fancy for. User can edit all objects of map and export it as an image for the purpose of reports.

Maps yielded from domain specific language (DSL) are presented using two layouts :

  • Network Layout : displays the network context around each containers (datacenters, routing areas, subnets) ...
  • Tree Layout : a free disposition of all containers in the map with algorithm to minimize interweaving...

Finally the Ariane Taitale objects color are coming from support team color attached to container or application color which is attached to node. These colors are defined in Directories


RabbitMQ is an open source message-oriented middleware software which implements the Advanced Message Queuing Protocol (AMQP). RabbitMQ plugin basically extends the Ariane Core Directory and add RabbitMQ injectors intelligence into Ariane (responsible for getting the data from your RabbitMQ infrastructure, transforming them and pushing them into the Mapping DB).

Mapping RabbitMQ infrastructure will permit RabbitMQ users (IE: application developers) and operators to share running configuration in a efficient human readable manner. 


Ariane Rabbitmq Injector is divided into three submodules as listed bellow.


It extends the Ariane Core Directories entity by adding RabbitMQ components definitions you may deploy on your information system :

  • RabbitMQ nodes where you define URL and authentication to access to the internal RabbitMQ configuration 
  • RabbitMQ cluster (which is just the definition of the cluster name and a RabbitMQ node set)

JSON Parser

This module is responsible to get the information available from the RabbitMQ node REST API. It is developed with Groovy.


Injector module encapsulates Gears and Components objects which are responsible to mine the data coming from the RabbitMQ components and transform them. 

Each RabbitMQ injector gears are defined to interact with one data-source only to avoid multiple data-source lock into one thread. So we have the following kind of gears :

  • Directory gear which is responsible to look at directory to check if there is new RabbitMQ instances (cluster or single node) to analyse. This gear is responsible to start Component gears. Period between two directories check is configurable.
  • A Component gear is responsible to get configurations data of a RabbitMQ instance and push the same to persisted cache (Infinispan - local file) through Component object. Each component object have several fields describing the data sniffed from RabbitMQ but an important point to keep in mind here is that there are two fields for the same RabbitMQ configuration data : 
    • one field for the last sniff
    • one field for the new sniff
    That way we're simplifying a lot the differences algorithm between the two sniffs. Period between two sniffs is configurable. When sniff is done the injector gears are sending message to the messaging gear (through Akka actors).
  • Mapping gear which is liable to get data coming from the sniffing, do the diff between last sniff and new sniff and transform the RabbitMQ configurations accordingly to fit the Messaging DB model.

Mapping Transformations

to fit our mapping needs we transform the RabbitMQ objects into supported mapping objects.

Here is the basic mapping transformation table : 

RabbitMQ ObjectMapping ObjectNotes
RabbitMQ ClusterCluster ! (smile) 
RabbitMQ Node


As an administrable component RabbitMQ Node is definitively a container (wink)
RabbitMQ VHostNode (owned by a RabbitMQ Node)[container:RabbitMQ Node] -owns-> {node:RabbitMQ VHost}
RabbitMQ QueueNode (owned by a RabbitMQ VHost){node:RabbitMQ VHost} -owns-> {node:RabbitMQ Queue}
RabbitMQ ExchangeNode (owned by a RabbitMQ VHost){node:RabbitMQ VHost} -owns-> {node:RabbitMQ Exchange}
RabbitMQ BindingEndpoint (owned by a RabbitMQ Exchange or Queue){node:RabbitMQ Queue}  -owns->  (endpoint:RabbitMQ Binding)


{node:RabbitMQ Queue}  -owns->  (endpoint:RabbitMQ Binding)

NOTE : queue and exchange are then linked through binding endpoints and memory link 
RabbitMQ Channel + Connection

Endpoint (owned by a RabbitMQ Queue
or Exchange or RabbitMQ Client Consumer or Producer)

Here we can extend what we see from a RabbitMQ node to define client container / consumer and producer.
We decided to merge RabbitMQ Channel + Connection as a endpoint to provide application links between these endpoints. eg :

[container:RabbitMQ Client process] -owns-> {node:RabbitMQ Client Consumer} -owns-> (endpoint:RabbitMQ Channel+Connection)
(endpoint:RabbitMQ Channel+Connection) <-owns- (node:RabbitMQ Queue) 


[container:RabbitMQ Client process] -owns-> {node:RabbitMQ Client Publisher} -owns-> (endpoint:RabbitMQ Channel+Connection)
(endpoint:RabbitMQ Channel+Connection) <-owns- {node:RabbitMQ Exchange}  

Finally, to link cluster node each other we are defining a specific cluster node with specific endpoints and links on each RabbitMQ node.



There are a lot of plugins to develop to get a complete map of the I.T world. Currently Echinopsii, the company promoting and supporting Ariane Framework can propose two plugins :

  • the Ariane RabbitMQ plugin
  • the Ariane TibcoRV plugin

One of our main mission is to write more plugins with the community help. Two other plugins are in the pipe and planned to be delivered this year : 

  • the Ariane Docker plugin
  • the Ariane Cassandra plugin

Meanwhile we're working on new plugins and also improving the Ariane Core framework features. We also have many ideas to continue to improvize the core framework :

  • adding the temporal dimension to our map system which will then give Ariane the ability to (re) play video on what happened to the I.S.
  • improve our Ariane Mapping DSL to add more intelligence behind each word pushed by the end user
  • improve the Ariane Taitale graph render to give some new cool U.X features
  • add some monitoring features to be merged with the Ariane On Demand map 
  • ....

Infrastructure System Mapping is a new exciting area of the information techonology and this is really easy to find new features to add on a map like Google did with Google map... So stay tuned (wink)




[1] Ann Rockley and Charles Cooper. Managing enterprise content. A unified content stragegy (2002). [Online] Available:
[2] Ben Kepes. Puppet and EMC Partner to bring DevOps to the enterprise (2014). [Online] Available: 
[3] John Minnihan. Why Docker + Ansible Make Chef and Puppet 100% Unnecessary (2014). [Online] Available :
[4] Corey Quinn. A taste of salt. Like Puppet, But Less Frustrating (2013) . [Online] Available :
[5] Ben Kepes. Chef Announces Partnerships And DevOps Workflows (2015). [Online] Available :
[6] Julie Arsenault. Reducing your incident resolution time (2014). [Online] Available : 
[7]  Lisa Quast. Why knowledge management is important to the success of your company (2012). [Online] Available: 
[8] John J Balena. DevOps Best Practices: Break Down Silos, Avoid Drift and Optimize for Flow (2014). [Online] Available :
[9] Echinopsii. Echinopsii Manifesto (2014). [Online] Available: 
[10] Echinopsii. Ariane project homepage (2014). [Online] Available: http://
[11] Eclipse Foundation. Eclipse Virgo web page. [Online] Available:
[12] OSGI Alliance. OSGi alliance homepage. [Online] Available: http://
[13] Martin Fowler. Monolith first. (2015) [Online] Available:
[14] RedHat. Infinispan web site. [Online] Available: 
[15] RedHat. Hibernate web site. [Online] Available:
[16] Neo Technology. Neo4J web site. [Online] Available:
[17] Typesafe. Akka web site. [Online] Available:
[18] EFPL. Scala web site. [Online] Available:
[19] The Groovy Project. Groovy web site. [Online] Available: 
[20] PrimeTek. PrimeFaces web site. [Online] Available:
[21] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Security (IDM). [Online] Available:
[22] Les Hazlewood. The new RBAC: resource base. (2012) [Online] Available:  
[23] MariaDB Foundation. MariaDB web site. [Online] Available: 
[24] Apache Foundation. Shiro web site. [Online] Available:
[25] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Portal. [Online] Available: 
[26] RedHat. RESTEasy web page. [Online] Available: 
[27] Oracle. Java Server Faces technology. [Online] Available: http://
[28] Echinopsii / Mathilde Ffrench. [Online] Available:
[29] Oracle. Java Persistence API. [Online] Available: 
[30] Echinopsii / GitHub branch of Hibernate ORM allowing extension of EntityManager entities set at runtime with OSGi. [Online] Available: 
[31] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Mapping. [Online] Available:
[32] Tinkerpop. Tinkerpop Blueprints home page. [Online] Available: 
[33] Dimitri Baranovskiy. Raphael javascript library homepage. [Online] Available: http:// 
[34] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Injector. [Online] Available:
[35] Pivotal. RabbitMQ homepage. [Online] Available:
[36] OASIS. AMQP homepage. [Online] Available:
[37] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Mapping / Abstract domain model. [Online] Available:
[38] Echinopsii / Mathilde Ffrench. Ariane Documentation / Development Guide / Ariane Components Guide / Mapping / Mapping DSL (mdsl). [Online] Available: 





Neo4J is a great Graph Database and OSGI is currently the most serious modularisation framework for Java. Unfortunately Neo4J is not OSGI friendly out of the box and consequently we need to repackage it. 

Have a clean integration of Neo4J on OSGI runtime is a major point for the Ariane project as in the long term Ariane will be composed by plenty of OSGI modules and µ-services. For the OSGI value proposition let's have a look to this and this

There is a lot of OSGI runtime and most of them are working well. We decided to go with EclipseVirgo because we failed to make work Primefaces on at the time we have to choose our OSGI container. Since then Apache Karaf seems have provided a correction on that subject but I never get time to test it again. So I'm not saying Eclipse Virgo is better than Apache Karaf. Eclipse Virgo is just working well for our current needs on the Ariane project. But finally the work done here with Eclipse Virgo should be reusable on Apache Karaf. Some little work will be necessary but not so much. Let me know if you're planning to do that work and I'll be happy to help you.

Target of this blog post is show how to integrate the orginal Neo4J community distribution on the Eclipse Virgo Kernel. Main idea here is to make work Neo4J in Eclipse Virgo with a minimal work and the minimal overhead : that's why we choose Eclipse Virgo Kernel and not Eclipse Virgo Tomcat/Jetty. Finally at the end of this blog post you'll be able to run Neo4J on an OSGI environment as if you run it from the official distribution.




Prepare Virgo Kernel to embed Neo4J

We first need to changes some Virgo Kernel configurations. So if not done already you have to download Virgo Kernel (last version is 3.6.3) and unzip it in some working directory. Let's call now $VIRGO_HOME the installed Virgo directory. You also have to get and install some JRE 1.7 to run Virgo.

Setup Virgo Java 1.7 environment

First of all you need to enable JavaSE 1.7 for Virgo: edit the $VIRGO_HOME/configuration/java6-server.profile and add JavaSE-1,7 to the org.osgi.framework.executionenvironment list :

diff configuration/java6-server.profile

Setup Virgo SSH

To deploy OSGI bundles or Virgo plans|pars, we will use the Virgo SSH console. You need to enable the Virgo SSH consoles by editing following files :

  • $VIRGO_HOME/configuration/ for the virgo kernel region
  • $VIRGO_HOME/repository/ext/ for the virgo user region

Virgo Regions


To know more about Virgo regions let's have a look to the Virgo documentation.

diff configuration/
diff repository/ext/

Virgo users configuration


You can connect on Virgo through SSH by using the default user/password : admin/springsource.
You can configure the Virgo users through following file : $VIRGO_HOME/configuration/
To know more about authentication in Virgo let's have a look to Virgo documentation


Setup Virgo neo4j-community repository

Finally we setup a dedicated neo4j-community repository into Virgo by :

  • creating the $VIRGO_HOME/repository/neo4j-community directory
  • editing the $VIRGO_HOME/configuration/ file

diff configuration/

External vs Watched Virgo repository


You may want to configure a watched repository instead of an external one (Virgo will then load changes on the repository at runtime). To know more about Virgo provisioning repository look at the Virgo documentation.



Install Neo4j into Eclipse Virgo Kernel

Neo4J community classpath

When you unzip the Neo4j community distribution you have two directories where you can find Neo4J JARs which are composing the Neo4J runtime classpath :

  • $NEO4J_HOME/system/lib
  • $NEO4J_HOME/lib

There is a lot of non ready OSGI JARs in the Neo4J 2.1.2 community distribution and most of them are coming from Neo4J JARs. We have different ways to manage these non OSGI friendly JARs we will choose depending on the situations. Our strategy is resumed at the end of this blog post.


Install OSGI ready Neo4j dependencies into Virgo

Neo4J community dependencies will be installed in $VIRGO_HOME/repository/neo4j-community. All these bundles are public bundle you can find on maven repos (eg :


List of primary Neo4J community dependencies to install


To makes some of these bundles happy in OSGI environment you also need to add others bundles to fit the dependencies.

  • org.apache.commons.configuration 1.6.0 needs javax.mail_1.4.0.v201005080615.jar.
    I installed this one in $VIRGO_HOME/repository/ext as this is the target repository for this bundle in the Virgo Tomcat distribution.
  • org.apache.commons.configuration 1.6.0 needs commons-codec_1.5.jar.
    I installed this one in $VIRGO_HOME/repository/ext as this is the target repository for this bundle in the Virgo Tomcat distribution.
  • org.apache.commons.configuration 1.6.0 needs commons-jxpath-1.3.jar.
    I installed this one in $VIRGO_HOME/repository/neo4j-community.
  • org.apache.commons.jxpath 1.3.0 needs javax.servlet.jsp_2.2.0.v201112011158.jar
    I installed this one in $VIRGO_HOME/repository/ext as this is the target repository for this bundle in the Virgo Tomcat distribution.
  • javax.servlet.jsp 2.2.0.v201112011158 needs javax.el_2.2.0.v201108011116.jar
    I installed this one in $VIRGO_HOME/repository/ext as this is the target repository for this bundle in the Virgo Tomcat distribution.
  • org.apache.commons.jxpath 1.3.0 needs
    I installed this one in $VIRGO_HOME/repository/neo4j-community.


Finally after these first dependencies install repositories look like this :







Transform non OSGI friendly JARs dependencies into OSGI bundles 

There is some Neo4J dependencies which are not OSGI friendly JARs and we need to transform them as OSGI bundles. That transformation implies defining an OSGI ready manifest into these JARs without touching any line of codes. 
This task could be very boring but Eclipse Virgo offers us a cool tool which produces relevant OSGI manifest from a JAR file : Eclipse Virgo Bundlor

To make our life more easier, we provide some scripting around this tool and a collection of bundlor manifests in our osgi factory. This is the way we created the following bundles from original JARs to install into Virgo Kernel neo4j-community repository.
Actually most of the bundles generated through the echinopsii osgi factory can be downloaded from the echinopsii maven repository


Original JAR nameBundle nameBundlor manifestComments

go to manifest

This bundle has been regenerated from the original one to declare some dependencies as optional (like tomcat dependencies)

go to manifest

Neo4J original dependency as a bundle

go to manifest

Neo4J original dependency as a bundle

go to manifest

Needed dependency for janino-2.6.1

go to manifest

Needed dependency for janino-2.6.1

go to manifest

Needed dependency for janino-2.6.1

go to manifest

Neo4J original dependency as a bundle

go to manifest

Neo4J original dependency as a bundle


Once installed into Virgo, the neo4j-community repository looks like this : 




The Neo4J community super-bundle

We just saw how we can easily transform non OSGI friendly jar into OSGI bundle with Eclipse Virgo Bundlor. But some times this is not enough to make work non-osgi friendly JARs into OSGI environment :

  • because you may have import constraint which becomes spaghetti-like dependency chain which could be horrible to demystify. This is the case of the jersey dependency chain.
  • because of the heavily use of java ServicesLoader feature introduced in Java 6 - which is not really OSGI friendly. This is the case of the Neo4J libraries.

In such cases the quickest (as you decide to NOT reverse-engineer the dependencies) and safest (as you decide to NOT add any code lines) solution is to create a super-bundle which will include all the problematic JARs and therefore provides a same classpath area for all these dependencies you want to make work together.

To make this super-bundle we use the great work from Apache Servicemix team who deliver a maven engineering tool to generate OSGI bundle from severals dependencies. The Neo4J community super-bundle definition can be found here.

The final repository looks like this : 



Install and start Neo4J super-bundle in Eclipse Virgo Kernel

Once Virgo Kernel is started ($VIRGO_HOME/bin/ we can connect on the Kernel Region console through SSH : 

Then we can start the Neo4J super-bundle like this : 

Examine Neo4J super-bundle  Expand source



Start the Neo4J server in Eclipse Virgo Kernel

Neo4J and Jetty integration into Virgo Kernel environment

To add the Jetty configuration into Virgo Kernel environment, create the $VIRGO_HOME/org/eclipse/jetty/webapp directory. Then copy the Jetty webdefault.xml in it.

To add the neo4j configuration into Virgo Kernel environment, create the following directories :

  • $VIRGO_HOME/neo4j/conf
  • $VIRGO_HOME/neo4j/data
  • $VIRGO_HOME/neo4j/graph

Then copy the neo4j configuration files into $VIRGO_HOME/neo4j/conf : 

You need to adapt these files to fit your environment path.

Enable Neo4J RMI in Virgo Kernel environment

Neo4J use RMI and so we need to enable it in Virgo Kernel environment. 
Edit the $VIRGO_HOME/bin/ and add the following lines : 

Add RMI to Virgo Kernel


Then add and edit $VIRGO_HOME/configuration/virgo.policy :

Virgo security policy

OSGI security


OSGI provides security features extending the Java ones. To know more loot at this doc and this presentation.
The above security policy is totally unsecure but very  convenient. Feel free to adapt this policy to your paranoïa...


Get and install the neo4j-osgi demonstration application

To start the Neo4J server in Virgo, we will use a dedicated application which bootstrap Neo4J following OSGI way. You can get this application here.


This little class is the OSGI bootstrapping handler. When starting/stopping the bundle the start/stop method of this class are called.
At startup the only thing done is the Neo4JManagedService registration into the service registry with the pid "net.echinopsii.demo.Neo4JManagedService".


The Neo4JManagedService handle the configuration and start the Neo4J server according to the neo4j configuration path.



When building the project, maven will setup the OSGI manifest of the bundle according to this pom. Look at the necessary import to make work the Neo4J bootstrap.



install the demo app

Once builded you can install the neo4j-osgi-1.0.0.jar in the $VIRGO_HOME/repository/usr. 
You also need to define the Neo4JManagedService configuration by adding and editing $VIRGO_HOME/repository/usr/ file : 

Neo4JManagedService configuration

Finally you can configure the log level of this little app in $VIRGO_HOME/configuration/serviceability.xml

Set up log level


Run neo4j-osgi demonstration application and test the OSGI embedded Neo4J server

Execute the following commands to make run the OSGI embedded Neo4J server.

Starting Neo4J server

Once these commands are executed you can see the following lines into the Virgo logs ($VIRGO_HOME/serviceability/logs/log.log)

And consequently you can have a look to the Neo4J webapplication (http://your-server-fqdn:7474) and begin to play with the Neo4J Movie Graph tutorial (smile)




Annex : Neo4J classpath bundleisation strategy

JARs in $NEO4J_HOME/system/libOSGI bundle ?Bundleisation methodology
asm-3.1.jarnouse already asm bundle by springsource
to be installed in Virgo 
bcprov-jdk16-140.jarnouse already bouncycastle bundle by springsource
to be installed in Virgo 
commons-beanutils-1.8.0.jaryesto be installed in Virgo
commons-beanutils-core-1.8.0.jarnosame as commons beanutils
commons-collections-3.2.1.jaryesto be installed in Virgo
commons-compiler-2.6.1.jarnobundleisation through bundlor
to be installed in Virgo 
commons-configuration-1.6.jaryesto be installed in Virgo
commons-digester-1.8.1.jaryesto be installed in Virgo
commons-io-1.4.jaryesprovided by Virgo :
commons-lang-2.4.jaryesto be installed in Virgo
commons-logging-1.1.1.jarnoprovided by Virgo :
jackson-core-asl-1.9.7.jaryesto be installed in Virgo
jackson-jaxrs-1.9.7.jaryesto be installed in Virgo
jackson-mapper-asl-1.9.7.jaryesto be installed in Virgo
janino-2.6.1.jarnobundleisation through bundlor
to be installed in Virgo 
javax.servlet-3.0.0.v201112011016.jaryesprovided by Virgo:
jcl-over-slf4j-1.6.1.jaryesprovided by Virgo :
jersey-core-1.9.jaryesNeo4j super bundle
jersey-multipart-1.9.jaryesNeo4j super bundle
jersey-server-1.9.jaryesNeo4j super bundle
jetty-http-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-io-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-security-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-server-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-servlet-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-util-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-webapp-9.0.5.v20130815.jaryesto be installed in Virgo
jetty-xml-9.0.5.v20130815.jaryesto be installed in Virgo

to be ignored
packages already present in jersey-core...

logback-access-1.1.2.jaryesto be installed in Virgo
(Note: we'll use logback-access-1.0.7 to fit Virgo logback versioning)
logback-classic-1.1.2.jaryesprovided by Virgo :
logback-core-1.1.2.jaryesprovided by Virgo : 
mimepull-1.6.jaryesto be installed in Virgo
neo4j-browser-2.1.2.jarnoNeo4j super bundle
neo4j-server-2.1.2.jarnoNeo4j super bundle
neo4j-server-2.1.2-static-web.jarnoNeo4j super bundle
opencsv-2.0.jarnoNeo4j super bundle
rhino-1.7R4.jarnobundleisation through bundlor
to be installed in Virgo 
rrd4j-2.0.7.jarnobundleisation through bundlor
to be installed in Virgo 
slf4j-api-1.6.2.jaryesprovided by Virgo :
JARs in $NEO4J_HOME/libOSGI bundle ?Bundleisation methodology
concurrentlinkedhashmap-lru-1.3.1.jaryesto be installed in Virgo
geronimo-jta_1.1_spec-1.1.1.jar yes

provided by Virgo :

lucene-core-3.6.2.jarnouse servicemix bundle
to be installed in Virgo 
neo4j-cypher-2.1.2.jarnoNeo4j super bundle
neo4j-cypher-commons-2.1.2.jarnoNeo4j super bundle
neo4j-cypher-compiler-1.9-2.0.3.jarnoNeo4j super bundle
neo4j-cypher-compiler-2.0-2.0.3.jarnoNeo4j super bundle
neo4j-cypher-compiler-2.1-2.1.2.jarno Neo4j super bundle
neo4j-graph-algo-2.1.2.jarnoNeo4j super bundle
neo4j-graph-matching-2.1.2.jarnoNeo4j super bundle
neo4j-jmx-2.1.2.jarnoNeo4j super bundle
neo4j-kernel-2.1.2.jarnoNeo4j super bundle
neo4j-lucene-index-2.1.2.jarnoNeo4j super bundle
neo4j-primitive-collections-2.1.2.jar noNeo4j super bundle
neo4j-shell-2.1.2.jarnoNeo4j super bundle
neo4j-udc-2.1.2.jarnoNeo4j super bundle
opencsv-2.0.jarnobundleisation through bundlor
to be installed in Virgo 
org.apache.servicemix.bundles.jline-0.9.94_1.jaryesto be installed in Virgo
parboiled-core-1.1.6.jaryesto be installed in Virgo
parboiled-scala_2.10-1.1.6.jaryesto be installed in Virgo
scala-library-2.10.4.jaryesto be installed in Virgo
server-api-2.1.2.jarnoNeo4j super bundle