I am learning about Apache Storm and the guide had long and winding instructions on how to install Apache Storm. I installed the single node using brew and it was a breeze. I did not had to spend time figuring out out if I was installing latest version.
Pre-requisites:
Make sure that you have ‘HomeBrew‘ installed. One reason that I prefer ‘brew’ is that I do not have to remember where things are installed and brew manages and fixes any updates that have been introduced. It also makes installation and uninstall super easy. Everything this is saved under
/url/local/Cellar.
Step 1: Install Zookeeper
‘ brew install zookeeper’
After the installation is complete, you will see this message.
To have launchd start zookeeper at login:
ln -sfv /usr/local/opt/zookeeper/*.plist ~/Library/LaunchAgents
Then to load zookeeper now:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.zookeeper.plist
Or, if you don’t want/need launchctl, you can just run:
zkServer start
I prefer not to launch zookeeper at login as it will sow down my mac login. So I prefer to start and stop Zookeeper using the command-
‘zkServer start/stop’
Step 2: Install ZeroMQ
‘brew install zeromq’
Step 3: Install Apache storm
‘brew install storm’
This above command will install everything in Cellar folder and will have a symlink in ‘/usr/local/opt/storm’ folder.
Step 4: Update the storm config file storm.yaml.
This file will be in /usr/local/opt/storm/libexec/conf folder.
After opening this file you will see that everything is commented out. So we will add the following.
storm.zookeeper.servers:
– “localhost”
# – “server2”
#
nimbus.host: “localhost”
nimbus.thrift.port: 6627
ui.port: 8772
storm.local.dir: “/Users/dinesharora/storm/data”
java.library.path: /usr/lib/jvm”
supervisor.slots.ports:
– 6700
– 6701
– 6702
– 6703
If you will notice that I have added “storm.local.dir”. Make sure that whatever folder you create, they have the right permissions because I initially added “/home/user/storm/data” and did not realize that you cannot create any directory under “/home” folder. This was causing issues when I started nimbus and supervisor.
Step 5: start zookeeper, nimbus, supervisor and ui
Below are the location on shell scripts. Start them in the given sequence
‘zkServer start’
‘/usr/local/opt/storm/libexec/bin/storm nimbus’
‘/usr/local/opt/storm/libexec/bin/storm supervisor’
‘/usr/local/opt/storm/libexec/bin/storm ui’
Step 6: Check that everything is running smoothly.
To do that run the below command.
‘jps’
The above command should give the following result.
Dineshs-MBP:bin dinesharora$ jps
5282 supervisor
5267 nimbus
5460 core
5735 Jps
4235 QuorumPeerMain
Step 7: Check the server index page. Access the url – ‘http://localhost:8772/index.html’.
If you are able to access the page then your installation is successful.
P.S. If you upgraded to new OSX – El Captain then you might have issues with brew itself. Do yourself a favor and reinstall brew again.
~ Stir a storm
