Sunday, June 8, 2014
Wednesday, March 5, 2014
Audio fingerprinting on Raspberry Pi (part 1)
A short guide on how to do some basic audio fingerprinting on a Raspberry Pi also known as “my quest to notify myself when the dryer is done”.
Lets start with the excellent Chromaprint library.
Download package from Chromaprint source
Build/install Python packages now:
Create a test WAV file 10 seconds in length of whatever audio sound/noise/whatever that you want to fingerprint. In this case I’m using my USB microphone plugged into the Pi since there is no onboard microphone or audio input (and no analog inputs either). You can also use the PyAudio package you installed to record the wav file.
Fingerprint your wave file (in python). Note: You can just as easily use ‘fpcalc’ in the shell to get a compressed or raw fingerprint from chromaprint libs.
If all went well you should not have seen any errors and your compressed and raw fingerprint should have been printed to the screen. If you fingerprint only had “AQAA” in it then something is wrong with your chromaprint library. The available libchromaprint packages along with the python-pyaudio package did not work for me at all and consistently resulted in no fingerprints. Instead building the custom chromaprint libs and then installing pyaudio via pip resulted in a working setup.
If you do have fingerprint errors make sure your wav file is not empty and if it’s not empty then try to do a clean build on the chromprint libs and re-install.
Note: The example ‘fpcalc’ won’t actually build from the 1.1 chromaprint libs. It throws some strange compiler errors that I couldn’t resolve.
I’m going to be recording 2 10 second files in a loop and that can be a bit hard on the SD card in the Raspberry Pi so instead I’m using the built in TMPFS location in /run so I don’t do a lot of unnecessary writes to the SD card.
That’s all for now, come back for part 2 where I wrap it all up into a single script that records in a thread while analyzing the previous recording and sending notifications when matches are found.
Lets start with the excellent Chromaprint library.
Build Chromaprint on Pi
Install pre-reqs: sudo apt-get install libboost1.50-all-dev ffmpeg libtag1-dev zlib1g-dev resample libresample1-dev cmake libffms2-dev
Download package from Chromaprint source
mkdir /opt/src
cd /opt/src
wget https://bitbucket.org/acoustid/chromaprint/downloads/chromaprint-1.1.tar.gz
tar -zxvf chromaprint-1.1.tar.gz
cd chromaprint*
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_EXAMPLES=OFF -DCMAKE_CXX_FLAGS="-Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s" -DBUILD_TOOLS=ON .
make
sudo make install
Python stuff
If you don’t already have PIP installed go grab it: sudo easy_install pip
Build/install Python packages now:
sudo pip install pyacoustid audioread
Create a test WAV file 10 seconds in length of whatever audio sound/noise/whatever that you want to fingerprint. In this case I’m using my USB microphone plugged into the Pi since there is no onboard microphone or audio input (and no analog inputs either). You can also use the PyAudio package you installed to record the wav file.
arecord -D plughw:1,0 -d 10 -r 44100 --channels=2 --format=cd /tmp/test.wav
Fingerprint your wave file (in python). Note: You can just as easily use ‘fpcalc’ in the shell to get a compressed or raw fingerprint from chromaprint libs.
import chromaprint
import acoustid
duration, fingerprint = acoustid.fingerprint_file('/tmp/test.wav')
fp_raw = chromaprint.decode_fingerprint(fingerprint)[0]
print "Compressed fingerprint: %s" % fingerprint
print "Raw fingerprint: %s" % fp_raw
If all went well you should not have seen any errors and your compressed and raw fingerprint should have been printed to the screen. If you fingerprint only had “AQAA” in it then something is wrong with your chromaprint library. The available libchromaprint packages along with the python-pyaudio package did not work for me at all and consistently resulted in no fingerprints. Instead building the custom chromaprint libs and then installing pyaudio via pip resulted in a working setup.
If you do have fingerprint errors make sure your wav file is not empty and if it’s not empty then try to do a clean build on the chromprint libs and re-install.
Note: The example ‘fpcalc’ won’t actually build from the 1.1 chromaprint libs. It throws some strange compiler errors that I couldn’t resolve.
I’m going to be recording 2 10 second files in a loop and that can be a bit hard on the SD card in the Raspberry Pi so instead I’m using the built in TMPFS location in /run so I don’t do a lot of unnecessary writes to the SD card.
That’s all for now, come back for part 2 where I wrap it all up into a single script that records in a thread while analyzing the previous recording and sending notifications when matches are found.
Friday, February 21, 2014
Multi-master Puppet setup with DNS SRV records
I recently switched from a single Puppet master to a Multi-master setup which consists of a single CA server and 2 new masters. During this change I also took the time to upgrade my clients (and server) from 2.7.25 to 3.4.2 and Ruby from 1.8.7 to 2.0.0p353 oh and I also switched over to the new DNS SRV records setup.
While I'm not going to rehash the entire setup and all steps taken to get there I do want to include some high level steps that were not exactly clear after reading the Puppet labs docs on multi-master setups. Hopefully this helps others trying to accomplish the same thing.
For my setup I used a standalone CA and 2 masters. Follow the Puppet labs guides/docs to build out your CA and masters with whatever software you like. I used the blessed Apache + Passenger setup. For multi-masters there is no special setup required on the Apache/Passenger side of things, just set them up as usual with the exception of your config.ru file (see below).
If you are upgrading from a single master 2.X setup you also need to remove any $servername references from your manifests. Most likely this will be in manifests/site.pp file.
Don't just copy your config.ru from your old setup if you ran Puppet 3.x or older.
Use the new config.ru.passenger.3 in the Puppet labs github repo. If you skip this step you will have a series of odd problems that you won't be able to resolve any other way. Make sure you chown the config.ru file as puppet:puppet since Passenger uses the owner of the file as the user to run as.
Before you start your new CA or master servers you have to generate the SSL certs properly.
On the CA: make sure your /etc/puppet/puppet.conf contains the lines below (adjust config as needed to support your setup):
Verify that your cert looks correct with the command below, it should list your puppetca plus the alternate DNS names you specified.
Once that cert is signed you can hit CTRL-C and stop your master, now start it back up using the real web service. Once again check the weblogs for any errors. Try running the puppet agent by hand on this master now and see how it goes. You should get a clean run.
Now head over to your 2nd or 3rd master and repeat the steps above for the masters.
With your masters and your CA server working you can now tackle the clients.
Using your existing puppet master (if you have one) add all the lines in the [main] section above to your clients. You can safely do this ahead of time because the 2.X clients don't support those features and will just ignore them.
Now upgrade your packages via whatever tools you use to do package upgrades, for my setup I have a custom build of Ruby 2.0 packaged as an RPM using a fairly standard SPEC file.
I then used the FPM utility to package up Puppet, Facter and all dependancies (don't forget about Augeas if you use it).
Now on my hosts I can do a 'yum install ruby20-puppet' and everything gets upgraded. Make sure your Puppet.conf file has those srv_domain bits above and then delete your clients 'ssl' directory. Run the agent, it should automatically switch over to the new CA and masters and generate a cert, go sign it (or turn on autosigning), once signed the client should finish it's run as usual.
One final note: Currently Puppet pluginsync is broken with 3.4.2 (and below) when using DNS SRV records. This should be fixed in a later version but the simple workaround for now is to remove the implied $servername portion in pluginsync and instead let it use the server that the client connected to by putting this line in each and EVERY puppet.conf file for both agents and masters in the [main] section.
While I'm not going to rehash the entire setup and all steps taken to get there I do want to include some high level steps that were not exactly clear after reading the Puppet labs docs on multi-master setups. Hopefully this helps others trying to accomplish the same thing.
For my setup I used a standalone CA and 2 masters. Follow the Puppet labs guides/docs to build out your CA and masters with whatever software you like. I used the blessed Apache + Passenger setup. For multi-masters there is no special setup required on the Apache/Passenger side of things, just set them up as usual with the exception of your config.ru file (see below).
If you are upgrading from a single master 2.X setup you also need to remove any $servername references from your manifests. Most likely this will be in manifests/site.pp file.
Don't just copy your config.ru from your old setup if you ran Puppet 3.x or older.
Use the new config.ru.passenger.3 in the Puppet labs github repo. If you skip this step you will have a series of odd problems that you won't be able to resolve any other way. Make sure you chown the config.ru file as puppet:puppet since Passenger uses the owner of the file as the user to run as.
Before you start your new CA or master servers you have to generate the SSL certs properly.
On the CA: make sure your /etc/puppet/puppet.conf contains the lines below (adjust config as needed to support your setup):
[main]
pluginsource = puppet:///plugins
pluginsync = true
use_srv_records = true
srv_domain = mydomain.com
[master]
ca = true
dns_alt_names=myca1.mydomain.com,myca1
# Bits for Passenger/Apache
certname=puppetca.mydomain.com
ssl_client_header=SSL_CLIENT_S_DN
ssl_client_verify_header=SSL_CLIENT_VERIFY puppet cert generate puppetca.mydomain.com --dns_alt_names=myca1.mydomain.com,myca1 Verify that your cert looks correct with the command below, it should list your puppetca plus the alternate DNS names you specified.
puppet cert list puppetca.mydomain.com
[main]
pluginsource = puppet:///plugins
pluginsync = true
use_srv_records = true
srv_domain = mydomain.com
[master]
ca = false
# Bits for Passenger/Apache
certname=master1.mydomain.com
ssl_client_header=SSL_CLIENT_S_DN
ssl_client_verify_header=SSL_CLIENT_VERIFY
puppet master --no-daemonize --verbose
Once that cert is signed you can hit CTRL-C and stop your master, now start it back up using the real web service. Once again check the weblogs for any errors. Try running the puppet agent by hand on this master now and see how it goes. You should get a clean run.
Now head over to your 2nd or 3rd master and repeat the steps above for the masters.
With your masters and your CA server working you can now tackle the clients.
Using your existing puppet master (if you have one) add all the lines in the [main] section above to your clients. You can safely do this ahead of time because the 2.X clients don't support those features and will just ignore them.
Now upgrade your packages via whatever tools you use to do package upgrades, for my setup I have a custom build of Ruby 2.0 packaged as an RPM using a fairly standard SPEC file.
I then used the FPM utility to package up Puppet, Facter and all dependancies (don't forget about Augeas if you use it).
Now on my hosts I can do a 'yum install ruby20-puppet' and everything gets upgraded. Make sure your Puppet.conf file has those srv_domain bits above and then delete your clients 'ssl' directory. Run the agent, it should automatically switch over to the new CA and masters and generate a cert, go sign it (or turn on autosigning), once signed the client should finish it's run as usual.
One final note: Currently Puppet pluginsync is broken with 3.4.2 (and below) when using DNS SRV records. This should be fixed in a later version but the simple workaround for now is to remove the implied $servername portion in pluginsync and instead let it use the server that the client connected to by putting this line in each and EVERY puppet.conf file for both agents and masters in the [main] section.
[main]
pluginsource = puppet:///plugins
Subscribe to:
Posts (Atom)