amqnotifyfs

ActiveMQ Notification Filesystem

Warning The software in this article is highly outdated and unsupported. It might still work, I don’t know and I don’t care. And it was simply a proof of concept from the start.

Description

AMQNotifyFS is a fuse-based filesystem capable of sending simple file-changed notifications over an ActiveMQ bus.

Warning

This software has NOT been tested (barely in a test setup, as a proof of concept).

For example, this won't attempt to recover in case of errors. I made this for fun...

Why?

There are already several ways to be notified of file changes, why create a filesystem to do it?

Well.. there's dnotify, through famd. It's slow, file descriptor hungry, etc... It was replaced by inotify.

Then there's inotify. Better than dnotify on several points, but still has at least one huge drawback: you have to register every directory to monitor the files directly under it. If you want to monitor an entire tree, you have to register every subdirectory yourself.

Now imagine you have a big tree containing 150000 files distributed into a complex directory structure of 20000 directories and subdirectories. You have to scan the whole tree and register all those directories, and this can take quite some time, both for the scan and the registering.

Now if a new directory gets created.. you have to add it to the monitored list.

Enter amqnotifyfs! You mount it over the tree to monitor and BAM!, everything under it is monitored.

Monitored system calls

  • mknod
  • mkdir
  • unlink
  • rmdir
  • symlink
  • rename
  • link
  • chmod
  • chown
  • truncate
  • release (file close for files opened as write-only or read/write)

Messages

The messages sent are very simple, XML-based.

Example file or dir removal message:

<?xml version="1.0" encoding="UTF-8"?><fsrm><path>FILEPATH</path></fsrm>

Actually, the FILEPATH is always sent enclosed in a CDATA. Maybe it'll eventually be base64-encoded.

For a file/dir update/creation, the message is the same except the enclosing element is named "fsnew".

How to install

$ ./configure
$ make
$ sudo make install

How to use

See man page, source code, ...

For broker-uri and topic/queue usages, see ActiveMQ-CPP's web page, specifically the "Configuring" section.

Example

$ amqnotifyfs \
    '--broker-uri=tcp://127.0.0.1:61616?wireFormat=openwire&amp;transport.useAsyncSend=true' \
    --topic=TEST.FOO \
    /a /b

Monitors all file modifications in /b and sends notifications on the topic TEST.FOO to ActiveMQ, running on localhost. The real files are stored in /a.

Download

Get AMQNotifyFS: