IBM did a world leadership publish on SPECjEnt  benchmark that was accepted by the SPEC organization.  The result are available here:  http://www.spec.org/jEnterprise2010/results/res2012q2/.

If you are a WAS customer that needs to squeeze the last bit of performance out of WAS on a performance mission critical application then you should study and emulate in DETAIL all the custom parameters and tuning done by IBM on the application server to get this world class performance of 524 enterprise job transactions per second. See the entire publish here

This benchmark heavily stresses the JPA, web container and web services containers in WAS. After going through the results I discuss the relevant server side tuning below -

Application Server & JVM Tuning:  (*Your mileage may vary)

-Xms2048m

-Xmx2048m 

-Xmo470m 

-Xmn1578m 

-Xss128k 

-Xgcpolicy:gencon 

-Xgcthreads3 

-Xnoloa
-Xshareclasses:none 

-Xaggressive 

-Xtrace:none 

-Xlp
-Xdisableexplicitgc 

-Dcom.ibm.CORBA.FragmentSize=10000 

-Djavax.xml.transform.TransformerFactory=org.apache.xalan.processor.TransformerFactoryImpl
-Djavax.xml.parsers.SAXParserFactory=org.apache.xerces.jaxp.SAXParserFactoryImpl
-Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
-Djava.net.preferIPv4Stack=true 

-Dsun.net.inetaddr.ttl=0 

-DdisableWSAddressCaching=true
-Dcom.ibm.websphere.webservices.http.connectionKeepAlive=true 

-Dcom.ibm.websphere.webservices.http.maxConnection=50
-Dcom.ibm.websphere.webservices.http.connectionIdleTimeout=6000 

-Dcom.ibm.websphere.webservices.http.connectionPoolCleanUpTime=6000
-Dcom.ibm.websphere.webservices.http.connectionTimeout=0

APAR Fixes 

PM61650 

PM61561 

PM61599

Hardware Tuning for maximum Application Server performance  (*Your mileage may vary)

fs.file-max = 1048576
kernel.sem = 250 32000 100 128
kernel.shmall = 4294967296
kernel.shmmax = 68719476736
net.core.netdev_max_backlog = 250000
net.core.optmem_max = 30000000
net.core.rmem_default = 30000000
net.core.wmem_default = 30000000
net.core.rmem_max = 32554432
net.core.wmem_max = 32554432
net.core.somaxconn = 32767
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem  = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 20480 174760 32554432
net.ipv4.tcp_wmem = 20480 174760 32554432
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
vm.nr_hugepages = 8192
ulimit -n 1048576

Network Tuning:
  set txqueuelen 60000
  Stop irqbalance.
  Network interrupts bound to cores corresponding to the server instances.

Server instances were started using taskset, 2 instances per chip.

The WebSphere PMI  infrastructure provides a window (metrics) into the runtime state of the WebSphere Application Server. General overhead of basic PMI is roughly 2%. Your mileage may wary depending on which modules you enable when using Custom PMI. Please see attached document for the critical PMI counters in WAS. Key PMI modules in WAS

Recently we have improved the performance and footprint of session serialization and deserialization code when failover occurs in WAS.  Issues like session replication causing  OOM have been fixed by APAR 

PM38550: OUTOFMEMORY DURING FAILOVER WITH MEMORY-TO-MEMORY SESSION REPLICATION. 

When the fix is enabled by setting the JVM custom property DRS_BATCH_INTERVAL_SIZE to 50.  The DRS service will replicate HTTP session data in multiple batches of 50 objects in each batch. 

Abstract:  SSL connections between the plug-in and WebSphere Application Server might fail or revert to non-SSL after the shipped version of the plugin-key.kdb password expires April 26, 2012 US EDT.  Note:  If you are using the WebSphere Key and Certificate Management generated plug-in key store you are NOT affected.   

CVE ID: CVE-2012-2162
CVSS:
CVSS Base Score: 6.8
CVSS Temporal Score: See http://xforce.iss.net/xforce/xfdb/74900 for the current score
CVSS Environmental Score*: Undefined
CVSS String: (AV:N/AC:M/Au:N/C:P/I:P/A:P)
NEW FLASH can viewed at: hhttp://www-01.ibm.com/support/docview.wss?uid=swg21591172

Original Flash titled "Password to the plugin-key.kdb file expires on April 26, 2012 US EDT" may be viewed at:   http://www-01.ibm.com/support/docview.wss?uid=swg21588312

Basic Dynacache servlet caching will work with Struts 2; however Dynacache support for struts 2 is tricky since Struts2 is not backwards compatible with Struts1 and there is no ActionServlet in Struts 2.

Please note
- Dynamic caching may NOT always work with ALL the cachespec.xml combinations offered by Dynacache.
- Customer will need to set the  com.ibm.ws.webcontainer.invokefilterscompatibility  webcontainer custom property to true to get strut2 fragment caching to work.
- When caching Struts 2 output, the cache entries will need to be written  for the appropriate JSPs and servlet to be served by the application. 

With Java 6 the the IBM JIT  is aggressive in optimization leading to world class performance; however sometimes the JIT gets tripped up on itself doing certain optimizations.  If you ran into http://www-01.ibm.com/support/docview.wss?uid=swg1IZ58251 or http://www-01.ibm.com/support/docview.wss?uid=swg1IV15424 or http://www-01.ibm.com/support/docview.wss?uid=swg1IV16044 you may be wondering what is meant by JIT invariant argument optimization and if it makes sense to disable it ?

How does pre-existence work?
For a full and rigorous explanation see Chapter 5 from the paper that introduced the concept:

Assume the following scenario:

void m1(Foo arg)
{
   ...
   arg.m2(); // virtual call or interface call
   ...
}

If at the time of compilation for m1 there is only one implementation for callee m2, then the JIT knows exactly what target to expect and can safely inline the code for it with no guard.
If a class load event extends the hierarchy such that a new implementation for m2 is introduced into the system, the code for m1 may call the wrong target for some type of arguments. The JIT invalidates the old body for m1 and immediately schedules a recompilation for it. While the recompilation is in progress application threads are blocked from entering the old body of caller m1. The new body of m1 will not inline m2, but will actually execute the virtual/interface call (note that the different targets for m2 may already be compiled/optimized on their own). Pre-existence is not going to help in this case.

In a nutshell, pre-existence helps the JIT inline some targets when the class hierarchy indicates it is safe to do so.
1) If the class hierarchy is such that the JIT doesn't know what to inline, then pre-existence does nothing. With or without it it's the same thing.
2) If pre-existence was applied, but later-on the class hierarchy changed in an incompatible way, then the JIT basically needs to undo what pre-existence did. Again, disabling pre-existence from the beginning is not going to hurt this case.

Why have pre-existence enabled?
Some applications, while having deep class hierarchies, load only one implementer for various virtual calls. Experiments have shown improvements from pre-existence  Even when the class hierarchy is extended, we do not see any visible decrease in the performance of  the system due to method invalidation.

When does it make sense to disable pre-existence ?
When an application does a a lot of Dynamic class reloading and deep class hierarchies with interfaces having multiple implementations in mainline path, then it may make sense to disable pre-existence with

-Xjit:disableInvariantArgumentPreexistence generic JVM argument. 

Sometimes Eclipse Memory Analyzer (MA) our favorite tool for analyzing heap and system dumps dies to an OOM when opening LARGE heaps. When  opening these  large heapdumps ( system dumps or phd's > 4GB)  run Memory Analyzer with the following JVM options

-vm
/opt/IBM80/java/bin  
-Xms16000m   
-Xmx16000m   
-Xcompressedrefs        
-Xgcpolicy:gencon        

- These arguments are to be put in the eclipse.ini 
- Needs recent version of the JDK preferably the one under WAS (/opt/IBM80/java/bin)
- RAM on the machine needs to be at least the size of the JVM max heap on which the dump was taken.
- MA is NOT CPU bound. So any reasonably multi-core modern processor will do.
- We have seen times of up to 3 hours to parse phd heaps of around 4GB.
- For best performance heap analysis should be done as soon as it is opened otherwise the MA indexes which are maintained as soft references will be GCed and will need to be read back from the disk

I stumbled upon this google code project Dynacache L2 cache provider for Hibernate
http://code.google.com/p/webspherehibernateadapter/
http://visitgaurav.blogspot.com/2010/12/webspheres-dynacache-hibernate-caching.html

Gaurav has got most of the code right although there are some issues with the code which I list below -

 1. In org.hibernate.cache.dynacache.WASDynaCacheProvider.buildCache(String, Properties) you can create the cache programatically using com.ibm.wsspi.cache.DistributedObjectCacheFactory.getMap(String, Properties)
One of the properties passed in should have been the passed in regionName which directly maps to a cache instance in Dynacache. No need to do a JNDI lookup - 

DistributedObjectCacheFactory.createDistributedObjectCache(_cacheName, _cacheName,  _cacheInstanceProperties);

2. org.hibernate.cache.dynacache.WASDynaCache.asWASKey(Object) is unnecessary since Dynacache DistributedMap allows keys to be POJO and not just strings. Dynacache provides multiple cache regions which map 1:1 to the cache instance. There is no need to append the cache region to the cache key. Cache keys in two regions will never collide as they are separate cache instances.

3.  Dynacache memory cache instance size can be specified in # of entries as well as # of bytes on a 32 bit heap. The properties below can be used to control the cache instance size in bytes. Therefore org.hibernate.cache.dynacache.WASDynaCache.getSizeInMemory() could return the max cache size in MB. Unfortunately the external interfaces dont expose the current memory in bytes that the Dynacache implementation computes
http://webspherecommunity.blogspot.com/2008/09/new-dynacache-features-in-websphere-7.html
com.ibm.wsspi.cache.DistributedObjectCacheFactory.KEY_MEMORY_CACHE_SIZE_IN_MB
com.ibm.wsspi.cache.DistributedObjectCacheFactory.KEY_MEMORY_CACHE_SIZE_HIGH_THRESHOLD
com.ibm.wsspi.cache.DistributedObjectCacheFactory.KEY_MEMORY_CACHE_SIZE_LOW_THRESHOLD

4. Apply the following JVM custom properties to tune dynacache
http://wasdynacache.blogspot.com/2011/07/tuning-dynacache-and-drs.html

I have just pushed two very useful java programs/utilities that we have used in the past month for debugging file issues and searching for certain string patterns recursively in a directory.

https://github.com/kelapure/general-java-utils/blob/master/filewatcher/StringFinder.java
This program finds strings of an input  pattern recursively through a specified list of pattern matched directories.
Sort of like a java grep. We used this program to find untranslated messages that should have been internationalized.

https://github.com/kelapure/general-java-utils/blob/master/filewatcher/WatchDir.java
This  program will let you know if a particular directory is being written too. If any file in the directory is created, modified delated. This is very useful when you have to watch a particular directory structure for any changes.

Both these programs make extensive use of Java7  java.nio.file APIs for listing directories, files and for monitoring directories. Please modify and use for your needs.

http://www-01.ibm.com/software/webservers/appserv/was/performance.html

The IBM Health Center is the built in low cost , overhead and footprint profiler. The IBM Monitoring and Diagnostic Tools for Java - Health Center is a lightweight tool that monitors active IBM Virtual Machines for Java with minimal performance overhead. The Health Center suggests live tuning recommendations for garbage collection, profiles methods including call stacks, and highlights contended locks.

You should always upgrade to the latest version of health center server agent on your WAS JVM following this link -
http://publib.boulder.ibm.com/infocenter/hctool/v1r0/topic/com.ibm.java.diagnostics.healthcenter.doc/topics/installingagent.html




For more on monitoring a running Java applicaiton on the IBM JDK see
http://publib.boulder.ibm.com/infocenter/hctool/v1r0/topic/com.ibm.java.diagnostics.healthcenter.doc/topics/connecting.html

To load the health agent data you will need the IBM Health Center client bundled with the IBM Support Assistant .

Health Center agent tool in IBM Support Assistant

Sometimes there arise situations where certain JVM classes inside WebSphere Application Server like java.lang.Thread or java.util.Timer need to be instrumented to add some debug information. In such scnearios locate the src.zip file under WAS_HOME/java/src.zip. This file contains all the JDK source code. Extract the file you want to the eclipse workspace and make the necessary modifications like calling Thread.dumpStack() etc.,

Thereafter package the modified class file in its own jar say patch.jar and add the following property to the JVM generic argument
-Xbootclasspath/p:<FULLY_QUALIFIED_PATH_TO_JAR>


Note that the "/p" is important.  The bootclasspath is where the JDK loads "core" classes from, and the "/p" says to prepend the value you are supplying, thereby overriding the default location of the core classes.

Alternately


You can  configure the following WAS JVM generic argument to print  the stacktrace when a thread is started or stopped...
-Xdump:console:events=thrstart+thrstop

Sample output

JVMDUMP006I Processing Dump Event "thrstart", detail "" - Please Wait.
-------- Console dump --------


Stack Traces of Threads:


ThreadName=Signal Dispatcher(0805BFFC)
Status=Running


ThreadName=main(0805B5FC)
Status=Waiting
Monitor=0805ADE0 (Thread public flags mutex)
Count=0
Owner=(00000000)
        In com/ibm/oti/vm/BootstrapClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;
        In com/ibm/misc/SystemIntialization.lastChanceHook()V
        In java/lang/System.completeInitialization()V
        In java/lang/Thread.(Ljava/lang/String;Ljava/lang/Object;IZ)V


^^^^^^^^ Console dump ^^^^^^^^
JVMDUMP013I Processed Dump Event "thrstart", detail "".