Identify Java code consuming high CPU in Linux--linking JVM thread and Linux PID

The original article is in : http://blogs.manageengine.com/appmanager/2011/02/09/identify-java-code-consuming-high-cpu-in-linux-linking-jvm-thread-and-linux-pid/

 

We can easily identify a problematic java code which leads to high CPU utilization in Linux. Let me explain the steps with the following example,

1.  package test;
3.  public class ThreadDumpTest {
4.          public void test(){
5.                  for (int i = 0; i < 10 ; i++) {
6.                          Thread th=new Thread(new TR(i));
7.                          th.setName(“MyThread-”+(1000+i));
8.                          th.start();
9.                  }
10.          }
11.          public static void main(String[] args) {
12.                  ThreadDumpTest t=new ThreadDumpTest();
13.                  t.test();
14.          }
15.          private class TR implements Runnable{
16.                  int ins=0;
17.                  TR(int i){
18.                          ins=i;
19.                  }
20.                  public void run(){
21.                          while (true) {
22.                                  if(ins!=5) {
23.                                          try {
24.                                                  Thread.sleep(10000);
25.                                          } catch (Exception e) {
26.                                                  e.printStackTrace();
27.                                          }
28.                                  }
29.                          }
30.                  }
31.          }
32.  }

In the above example, all the threads are in while loop. Except ‘MyThread-1005 ‘ thread, all other threads will sleep 10 secs inside the loop. The ‘MyThread-1005 ‘ thread will not enter sleep part, so it will run in while loop without sleep. Due to while loop, the ‘MyThread-1005 ‘ thread will leads to high CPU utilization.

How to identify ?

Step 1 :

Execute ‘top ‘ command on the console. You can see the ‘java’ command with PID 7074 utilized 97% of CPU.

Step 2 :

The top command displays the process list like the above image. Press ‘Shift + h ‘ and wait few secs. You can see ‘Show threads on ‘ message in the top console. Now, you can see thread level details like CPU/Memory utilization. You can see a ‘java’ command thread with PID 7087 utilized 94% of CPU.

Step 3:

The identified problematic thread PID ( 7087 ) is in decimal format. Convert it into hexadecimal format . The respective hexadecimal for 7087 is 1BAF. And convert it into lowercase (1baf ).

Step 4:

 

Take thread dump and search the converted hexadecimal PID ( 1baf ) in the thread dump. You can find the hex PID as ‘nid ‘ (with ’0x’) of the thread trace. From the trace, you can find ‘MyThread-1005 ‘ is the problematic thread in the test program. And using the thread trace you can identify the problematic code and fix the problem.

1. [rameshj@rameshj thread]$ java test/ThreadDumpTest
2. ^\Full thread dump Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode):
3. “DestroyJavaVM” prio=1 tid=0x083d2d78 nid=0x1ba2 waiting on condition [0x00000000..0xbff4bf60]
4. “MyThread-1009″ prio=1 tid=0x08510a88 nid=0x1bb3 waiting on condition [0xa9cca000..0xa9ccb020]
5.         at java.lang.Thread.sleep(Native Method)                                             
6.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)          &nbs
   p;                     
7.         at java.lang.Thread.run(Thread.java:595)                                             
8. “MyThread-1008″ prio=1 tid=0×08510318 nid=0x1bb2 waiting on condition [0xa9d4b000..0xa9d4bea0]
9.         at java.lang.Thread.sleep(Native Method)                                             
10.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
11.         at java.lang.Thread.run(Thread.java:595)                                             
12. “MyThread-1007″ prio=1 tid=0x0850f298 nid=0x1bb1 waiting on condition [0xa9dcc000..0xa9dccf20]
13.         at java.lang.Thread.sleep(Native Method)                                             
14.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
15.         at java.lang.Thread.run(Thread.java:595)                                             
16. “MyThread-1006″ prio=1 tid=0x0850d818 nid=0x1bb0 waiting on condition [0xa9e4d000..0xa9e4dda0]
17.         at java.lang.Thread.sleep(Native Method)                                             
18.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
19.         at java.lang.Thread.run(Thread.java:595)                                             
20. “MyThread-1005 ” prio=1 tid=0x0850c9f8 nid=0x1baf runnable [0xa9ece000..0xa9ecee20]
21.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:43)                    
22.         at java.lang.Thread.run(Thread.java:595)                                 
23. “MyThread-1004″ prio=1 tid=0x0850b8e8 nid=0x1bae waiting on condition [0xa9f4f000..0xa9f500a0]
24.         at java.lang.Thread.sleep(Native Method)                                             
25.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
26.         at java.lang.Thread.run(Thread.java:595)                                             
27. “MyThread-1003″ prio=1 tid=0x0850aac8 nid=0x1bad waiting on condition [0xa9fd0000..0xa9fd1120]
28.         at java.lang.Thread.sleep(Native Method)                                             
29.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
30.         at java.lang.Thread.run(Thread.java:595)                                             
31. “MyThread-1002″ prio=1 tid=0x085099b8 nid=0x1bac waiting on condition [0xaa051000..0xaa051fa0]
32.         at java.lang.Thread.sleep(Native Method)                                             
33.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)                                
34.         at java.lang.Thread.run(Thread.java:595)                                             
35. “MyThread-1001″ prio=1 tid=0x08508a28 nid=0x1bab waiting
    on condition [0xaa0d2000..0xaa0d3020]
36.         at java.lang.Thread.sleep(Native Method)                                             
37.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)
38.         at java.lang.Thread.run(Thread.java:595)
39. “MyThread-1000″ prio=1 tid=0×08508520 nid=0x1baa waiting on condition [0xaa153000..0xaa153ea0]
40.         at java.lang.Thread.sleep(Native Method)
41.         at test.ThreadDumpTest$TR.run(ThreadDumpTest.java:45)
42.         at java.lang.Thread.run(Thread.java:595)
43. “Low Memory Detector” daemon prio=1 tid=0x0843cf58 nid=0x1ba8 runnable [0x00000000..0x00000000]
44. “CompilerThread0″ daemon prio=1 tid=0x0843b930 nid=0x1ba7 waiting on condition [0x00000000..0xaa3eb808]
45. “Signal Dispatcher” daemon prio=1 tid=0x0843a9d0 nid=0x1ba6 waiting on condition [0x00000000..0x00000000]
46. “Finalizer” daemon prio=1 tid=0×08430740 nid=0x1ba5 in Object.wait() [0xaa52c000..0xaa52d120]
47.         at java.lang.Object.wait(Native Method)
48.         – waiting on <0xaa6b0848> (a java.lang.ref.ReferenceQueue$Lock)
49.         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
50.         – locked <0xaa6b0848> (a java.lang.ref.ReferenceQueue$Lock)
51.         at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
52.         at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
53. “Reference Handler” daemon prio=1 tid=0x0842e898 nid=0x1ba4 in Object.wait() [0xaa5ad000..0xaa5adfa0]
54.         at java.lang.Object.wait(Native Method)
55.         – waiting on <0xaa6b0758> (a java.lang.ref.Reference$Lock)
56.         at java.lang.Object.wait(Object.java:474)
57.         at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
58.         – locked <0xaa6b0758> (a java.lang.ref.Reference$Lock)
59. “VM Thread” prio=1 tid=0x0842bb68 nid=0x1ba3 runnable
60. “VM Periodic Task Thread” prio=1 tid=0x0843e538 nid=0x1ba9 waiting on condition

Update : FYI : You can automatically take a thread dump of a Java application when a high CPU threshold is violated by using Applications Manager.  Learn more  or watch a related video or download Applications Manager now .