VisualVM Profiling Apache Tomcat through SSH Tunnel

So I had the job of trying to setup profiling of our Apache Tomcat instances on EC2.  There were a lot of instructions out there that I tried to cobble together for a solution.  The tricky part with profiling on a remote host are the firewall rules, since the RMI server will pick a random port to use.

The method I used was to open an SSH tunnel to the running machine and pretend like everything is local.  I finally got my monitoring up an running so wanted to share.

Props to Thiago for having a nice write-up which had details that many people ommitted.

1. First download the proper version of the catalina-jmx-remote.jar from the apache archives.  We were running on 7.0.23 so I downloaded: (you can just change the version on the link below)

2.  Copy the JAR to your Tomcat’s lib dir.

3.  Open server.xml in Tomcat’s conf dir (mine was /opt/tomcat7/conf/server.xml) and add the following listener.  You should have a group of Listeners in the file already so just add to the bottom:

<Listener className=”org.apache.catalina.mbeans.JmxRemoteLifecycleListener” rmiRegistryPortPlatform=”10001″ rmiServerPortPlatform=”10002″ useLocalPorts=”true” />

– The two ports listed here are arbitrary (you can pick your own).  By specifying both ports, you set which ports the RMI server uses instead of the server arbitarily picking them.

IMPORTANT: Many walkthroughs for SSH tunneling forget this part.  You need to set this to true.

4. Add the following params to startup:


– Many walkthroughs on the net tell you to set the “java.rmi.server.hostname” variable to the public IP.  But since we are using SSH tunneling this should be localhost.


5. Restart Tomcat and check to make sure that your Tomcat instance started with the params from step 4: ps -aux | grep tomcat

6.  Also verify that its listening on the 2 ports from Step 3: netstat -nlp


7.  Now you should be ready to connect with Visual VM.  Drop catalina-jmx-remote.jar into the VISUALVM_HOME/platform/lib directory.

8.  Create a SSH tunnel on BOTH ports.  Sub in your EC2 instance (and potentially change the user)

ssh N -L10001:localhost:10001 -L10002:localhost:10002 ec2-user@<EC2 public ip>

– Many instructions on the web only tell you to use 1 port, but you need both.

9.  Start VisualVM.

10.  Right-click on Local and and add a new JMX Connection and use the following service URL:




– Notice that the ports must match up with the ports you specified in Step 3.

And that should be it!  You should now be able to profile your application.




Remote Java Profiling on EC2

Had to do some profiling and there are a bunch of articles strewn across everywhere, so thought I would help the cause for people trying to find them. 

Props to Mriddle:

Another good article by Neil Figg:

I didn’t use Neils because I didn’t need to go deeper in my profiling at this stage.  I ran into a bunch of gotchas which Mriddle helped with and some comments:

1.  Create an SSH tunnel and make visual VM proxy through that:

– sudo ssh -i .ssh/yourkeyfileifyougotone.pem -D 9696

(options to add to jvisualvm)


2.  Command to check if jstatd is listening:

sudo netstat -nlp | grep jstatd

3.  Look at startup command for tomcat

ps aux | grep tomcat

4.  VisualVM looks at temp directories for process info, so make sure that you start jvisualvm and specify the proper temp dir (you can see it from step #3):

(option to add to jvisualvm)


Things that I read but didn’t work/didn’t try:

1.  Make sure the java bin you are using matches that on the remote server.  Apparently, there is a difference between the jre java bin and jdk java bin.


2.  Open the proper ports after your instance comes up on EC2.  Since jstatd picks a random port, you need to add the port to your security policy.


ec2-authorize <security_group> -p <port> -s <your ip>


My env on EC2:

OpenJDK Runtime Environment (IcedTea6 1.11.4) (amazon-

OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

Linux version 3.2.12-3.2.4.amzn1.x86_64 (mockbuild@gobi-build-31003) (gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC) )

Apache Tomcat/7.0.23


VisualVM: 1.6.0_33 (Build 110613); platform 110613-unknown-revn

Mac OS X (10.7.4) , x86_64 64bit

1.6.0_33; Java HotSpot(TM) 64-Bit Server VM (20.8-b03-424, mixed mode)