So I had the job of trying to setup profiling of our Apache Tomcat instances on EC2. There were a lot of instructions out there that I tried to cobble together for a solution. The tricky part with profiling on a remote host are the firewall rules, since the RMI server will pick a random port to use.
The method I used was to open an SSH tunnel to the running machine and pretend like everything is local. I finally got my monitoring up an running so wanted to share.
Props to Thiago for having a nice write-up which had details that many people ommitted.
1. First download the proper version of the catalina-jmx-remote.jar from the apache archives. We were running on 7.0.23 so I downloaded: (you can just change the version on the link below)
http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.23/bin/extras/
2. Copy the JAR to your Tomcat’s lib dir.
3. Open server.xml in Tomcat’s conf dir (mine was /opt/tomcat7/conf/server.xml) and add the following listener. You should have a group of Listeners in the file already so just add to the bottom:
<Listener className=”org.apache.catalina.mbeans.JmxRemoteLifecycleListener” rmiRegistryPortPlatform=”10001″ rmiServerPortPlatform=”10002″ useLocalPorts=”true” />
– The two ports listed here are arbitrary (you can pick your own). By specifying both ports, you set which ports the RMI server uses instead of the server arbitarily picking them.
– IMPORTANT: Many walkthroughs for SSH tunneling forget this part. You need to set this to true.
4. Add the following params to startup:
-Djava.rmi.server.hostname=localhost
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
– Many walkthroughs on the net tell you to set the “java.rmi.server.hostname” variable to the public IP. But since we are using SSH tunneling this should be localhost.
5. Restart Tomcat and check to make sure that your Tomcat instance started with the params from step 4: ps -aux | grep tomcat
6. Also verify that its listening on the 2 ports from Step 3: netstat -nlp
7. Now you should be ready to connect with Visual VM. Drop catalina-jmx-remote.jar into the VISUALVM_HOME/platform/lib directory.
8. Create a SSH tunnel on BOTH ports. Sub in your EC2 instance (and potentially change the user)
ssh N -L10001:localhost:10001 -L10002:localhost:10002 ec2-user@<EC2 public ip>
– Many instructions on the web only tell you to use 1 port, but you need both.
9. Start VisualVM.
10. Right-click on Local and and add a new JMX Connection and use the following service URL:
service:jmx:rmi://localhost:10002/jndi/rmi://localhost:10001/jmxrmi
– Notice that the ports must match up with the ports you specified in Step 3.
And that should be it! You should now be able to profile your application.
Kamil Mroczek 