/ IBM JVM tuning – gencon GC policy ~ Java EE Support Patterns

3.08.2012

IBM JVM tuning – gencon GC policy

This article will provide you detail on an important Java Heap space tuning consideration when migrating from a Java VM such as HotSpot or JRockit to the IBM JVM. This tuning recommendation is based on a recent troubleshooting and tuning mandate I performed for one of my IT clients.

IBM JVM overview

As you probably saw from my other articles, the IBM JVM is different than the HotSpot JVM in some aspects as it does not have a PermGen memory space for example. From a garbage collection perspective, it does provide you with advanced algorithms that can take advantage of a multi physical cores machine; similar to the HotSpot JVM.

From a troubleshooting perspective, IBM provides you with many tools; including out-of-the-box Thread Dump and Heap Dump generation capabilities from its JVM implementation.

The IBM JVM Thread Dump for example is particularly powerful as it provides you extra data on your JVM such as the active JVM environment variables, GC policies, loaded classes in each active class-loader etc. We will explore this in more detail on the part 4 of our Thread Dump Training plan.

IBM VM – default GC behaviour

Now back to our primary topic, it is very important that you understand the default behaviour of the IBM JVM garbage collector (version 1.5 & 1.6). By default, the Java Heap space is created using tenured memory only e.g. it does not create a separate YoungGen (nursery) space. This means that any memory allocation goes to the tenured space (short lived and long lived objects) which later gets collected by the default collector (via a Full GC).

** Please note that the IBM VM default GC policy has now been changed to gencon for JDK 1.7

Find below a verbose GC snapshot showing you the default GC memory breakdown with explanations:

<af type="tenured" id="5" timestamp="Mar 01 13:40:30 2012" intervalms="0.000">
  <minimum requested_bytes="48" />
  <time exclusiveaccessms="0.106" meanexclusiveaccessms="0.106" threads="0" lastthreadtid="0x000000011A846B00" />
  <tenured freebytes="20131840" totalbytes="2013265920" percent="0" >
    <soa freebytes="0" totalbytes="1993134080" percent="0" />
    <loa freebytes="20131840" totalbytes="20131840" percent="100" />
  </tenured>
  <gc type="global" id="8" totalid="2492" intervalms="2017588.587">
    <finalization objectsqueued="199" />
    <timesms mark="808.286" sweep="9.341" compact="0.000" total="818.292" />
    <tenured freebytes="1362331024" totalbytes="2013265920" percent="67" >
      <soa freebytes="1344212368" totalbytes="1995147264" percent="67" />
      <loa freebytes="18118656" totalbytes="18118656" percent="100" />
    </tenured>
  </gc>
  <tenured freebytes="1362330976" totalbytes="2013265920" percent="67" >
    <soa freebytes="1344212320" totalbytes="1995147264" percent="67" />
    <loa freebytes="18118656" totalbytes="18118656" percent="100" />
  </tenured>
  <time totalms="818.750" />
</af>



Ok, default IBM JVM GC policy is different… what is the problem?

The problem with this default JVM policy is that all Java objects are copied to the tenured space and collected via a global collection (Full GC). For many Java EE applications, the ratio of short lived vs. long lived objects is much higher. This means that your JVM will need to perform quite a lot of major collections to clean up the short lived objects; results: increased frequency of Full GC, increased JVM pause time, increased CPU utilization and performance degradation!

This is exactly what we observed while performing load testing following a migration to JVM HotSpot 1.5 (using incremental & parallel GC) to IBM JVM 1.6 with default GC policy. Heavy GC process was identified as the root cause as per the above explanation.

Solution please!

The good news is that the IBM JVM introduced generational & concurrent GC collector since version 1.5. This GC policy is providing exactly what we want:

-        It does split the Java Heap space between nursery and tenured spaces
-        Nursery (YoungGen) space objects are collected separately via the scavenger GC collector
-        Tenured space objects are collected via the global GC collector
-        The GC collector is concurrent and taking advantage of any multi physical cores machine

Results:

-        Reduced major collection frequency (Full GC)
-        Reduced Full GC elapsed time & pause time
-        Increase JVM throughput
-        Increase performance & capacity of your application

You can enable it by adding this JVM paremeter below:

-Xgcpolicy:gencon

Find below what you can expect in your verbose GC log after enabling this GC policy:

<af type="nursery" id="15" timestamp="Mar 08 05:34:06 2012" intervalms="1289096.227">
  <minimum requested_bytes="40" />
  <time exclusiveaccessms="0.085" meanexclusiveaccessms="0.085" threads="0" lastthreadtid="0x0000000118113900" />
  <refs soft="18043" weak="204057" phantom="27" dynamicSoftReferenceThreshold="32" maxSoftReferenceThreshold="32" />
  <nursery freebytes="0" totalbytes="530716672" percent="0" />
  <tenured freebytes="1887422016" totalbytes="2013265920" percent="93" >
    <soa freebytes="1786758720" totalbytes="1912602624" percent="93" />
    <loa freebytes="100663296" totalbytes="100663296" percent="100" />
  </tenured>
  <gc type="scavenger" id="15" totalid="15" intervalms="1289097.271">
    <flipped objectcount="1486449" bytes="129908000" />
    <tenured objectcount="1176" bytes="184144" />
    <finalization objectsqueued="3082" />
    <scavenger tiltratio="73" />
    <nursery freebytes="364304408" totalbytes="495208448" percent="73" tenureage="10" />
    <tenured freebytes="1886766656" totalbytes="2013265920" percent="93" >
      <soa freebytes="1786103360" totalbytes="1912602624" percent="93" />
      <loa freebytes="100663296" totalbytes="100663296" percent="100" />
    </tenured>
    <time totalms="233.886" />
  </gc>
  <nursery freebytes="364238872" totalbytes="495208448" percent="73" />
  <tenured freebytes="1886766656" totalbytes="2013265920" percent="93" >
    <soa freebytes="1786103360" totalbytes="1912602624" percent="93" />
    <loa freebytes="100663296" totalbytes="100663296" percent="100" />
  </tenured>
  <refs soft="17992" weak="5344" phantom="27" dynamicSoftReferenceThreshold="32" maxSoftReferenceThreshold="32" />
  <time totalms="236.858" />
</af>


Please keep in mind that it is still possible that your application may not benefit from this GC policy (bigger footprint of long lived objects etc.) so my recommendation to you is to always do your due diligence and perform proper capacity planning & load testing of your application before implementing any major tuning recommendations.

Conclusion

I hope this article has helped you understand the default IBM JVM 1.5/1.6 GC policy and how your Java EE application can benefit from this GC policy gencon tuning recommendation.

Please do not hesitate to post a comment or question at the end of this article. I’m also looking forward for your experience with the IBM JVM.

8 comments:

Hi, congratulations on your article. You're right that the gencon GC policy is often the best for the sort of transactional workloads seen in Java EE systems.

You might be interested to know that in Java 7 the default GC policy has been changed to gencon, so it's no longer necessary to specify this on the command-line.

Users migrating from Java 6 to Java 7 should perform proper capacity planning and load testing (as you advise) so they understand the different behaviour of the two policies. To revert to the Java 6 default policy you can use -Xgcpolicy:optthruput

Ian Partridge (IBM Java support team)

Thanks Ian for your productive comments,

I'm glad that you raised the fact that IBM Java 7 default GC policy is now gencon; very good decision here by the IBM Java team.

I will update this article with this extra detail so people have the right facts across all IBM JDK versions.

Thanks again.
P-H

Hi, and congratulations on your article too. I'm using gencon GC policy on a server, but when i study the JVM behaivour y see that the tenued free memory after GC goes down until 0. After that it occours a conn event: and the free tenued memory grows again. This actitude of the JVM machines is cyclically, and i don´t know if this behaivour of JVM its normal with this GC policy, or I have a memory leak, or something different.

Thanks anticipated.

William

Hi William,

If tenured space is not freed up after Full GC this can means either memory leak or Thread(s) consuming non stop memory at the time...

Can you please post a sample of your verbose GC output and I will review and analyze for you.

Thanks.
P-H

Good article P-H! Is '-Xgcpolicy:gencon' the only configuration I need to enable this on my server? Do I need to set other memory parameters such as -Xmn, etc?

Thanks,

Dennis Labajo (Delta Air Lines)

Hi Dennis,

GenCon is always a good starting point. Before making any further change, you will need to understand the Java heap memory footprint of your application in order to come up with a proper heap sizing.

Regards,
P-H

How can we understand which GC policy been configured in websphere application server (7.0.0.23). From WAS 8 default is generationalconcurrent(Gencon) . It might be optthroughput

Recently We are facing issues in our environment OOM. We kept 100MB(resaon being here for 100 MB is due to DB issue , when we increase minimum heap size more that 100 MB we are facing issue in DB paging space . and application is sending XML to store in DB )as minimum and 2GB as MAX heap . GC is continously cleanig objects and at some point of time in GC logs(nativestderr.log) says that "no free space available following GC" in expansion stanza . Please suggest me what can done done to tune application to come to normal stage
Env details : WAS 7.0.0.23
JDK:IBM 1.6

Post a Comment