Mirabelle 0.9.0: hello graphviz !
I released today the version 0.9.0 of Mirabelle, a stream processing engine I built. This release adds a new command named graphviz which generates a .dot file representing your streams configurations.
An example
Let’s take the commented stream example available on the Mirabelle main documentation page:
(streams
(stream {:name :multiple_branches :description "Example configuration"}
(where [:= :service "http_request_duration_seconds"]
(with :ttl 60
;; push everything into influxdb
(push-io! :influxdb)
;; index events in memory by host and service
(index [:host :service])
;; by will generate a branch for each :host value. Like that, downstream
;; computations will be per host and will not conflict between each other
(by [:host]
;; if the metric is greater than 1 for more than 60 seconds
;; Pass events downstream
(above-dt {:duration 60 :threshold 1}
;; pass the state to critical
(with :state "critical"
;; one alert only every 60 sec to avoid flooding pagerduty
(throttle {:duration 60 :count 1}
(push-io! :pagerduty)))))))))
The comments should be enough to explain what’s this stream is doing:
-
Filteting events with
:serviceequal tohttp_request_duration_seconds -
Setting the
:ttlfield to 60 if not set -
forwarding all events to InfluxDB and storing them into the stream index.
-
Then, we send an alert to Pagerduty if the duration is greater than 1 during 60 seconds.
Cool. Let’s now run the new graphviz subcommand: mirabelle graphviz /tmp/graph.dot. This .dot file can now be converted to png using dot -Tpng graph.dot > graph.png.
This is the content of graph.png

Another example
Lets now execute the same command on another example:
(streams
(stream {:name :main :default true :description "All events will arrive into this stream"}
;; inject events into the :alert and :influxdb streams
(reinject! :alert)
(reinject! :influxdb))
(stream {:name :alert :description "Send alerts to pagerduty"}
;; keep only critical events
(critical
;; split the actions by host/service
(by [:host :service]
;; let only one event pass every 60 sec
(throttle {:duration 60 :count 1}
;; send to pagerduty
(push-io! :pagerduty)))))
(stream {:name :influxdb :description "Send all events to InfluxDB"}
;; handle influxdb errors
(exception-stream
;; set the :source key to "mirabelle"
(with :source "mirabelle"
;; push to influxdb
(push-io! :influxdb))
;; set the :service key to "mirabelle-influxdb-error"
(with :service "mirabelle-influxdb-error"
;; inject events into the :alert stream
(reinject! :alert)))))
I have in this configuration 3 streams:
-
The first one named
:mainhas:default trueso it will be used by default for events arriving on Mirabelle and not targeting a specific stream. This stream will send all events (usingreinject!to the streams:alertand:influxdb. -
The second one named
:alertwill keep only events with:state = critical, and then send alerts to Pagerduty (with aby¨+throttleto avoid spamming Pagerduty too much: only one event will be send to Pagerduty every 60 seconds for each :host/:service combination). -
The third one named
:influxdbwill forward all events to InfluxDB. If pushing to InfluxDB fails (for example, the service is unavailable) an event containing the error will be sent to the:alertstream thank toexception-stream.
Let’s generate the graphical representation of these streams:

We can see how the streams are connected between each other.
Conclusion
I’m sure the .dot file generated by mirabelle graphviz can be improved to make it prettier, but I think this feature shows again the benefits of "compiling" the Mirabelle DSL into an intermediate datastructure (EDN) which can be then be consumed if needed (see the documentation for more information about this format).
I’m sure it should be easy for example to do the opposite: building a graphical editor for streams and then generating from the graphical representation the corresponding Mirabelle configuration !
Happy monitoring !
Add a comment
If you have a bug/issue with the commenting system, please send me an email (my email is in the "About" section).