All Things WebSphere
Concerns and issues relating to all versions of WebSphere Application ServerWednesday, August 31, 2011
wsadmin script to configure a data source for WebSphere Application Server
Do you know how to configure JDBC, DataSources, etc using wsadminlib.py library ?
See: https://cwiki.apache.org/ARCHIVA/archiva-on-websphere-61.html
Specifically, the file, createArchivaServer.py, from: https://cwiki.apache.org/ARCHIVA/archiva-on-websphere-61.data/createArchivaServer.py
And this is the relevant sections from the file:
# Specify the DB2 details.
db2DriverPath = "/usr/opt/db2_08_01/java"
db2Alias = "DB2ArchivaUserId"
db2UserId = "db2inst1"
db2Password = "db2inst1"
# JNDI details
db2DatabaseName = "ARCHIVA"
db2DriverType = "4"
db2ServerName = "localhost"
db2PortNumber = "50000"
and
# Set the WebSpere Variable needed to for the universal driver. In this case, we just set it for
# the individual server. To set it for the entire node, just remove the serverName variable completely.
print "Setting WebSphere Variable"
setWebSphereVariable( "DB2UNIVERSAL_JDBC_DRIVER_PATH", db2DriverPath, nodeName, serverName)
# Create a JAAS entry for the userid/password for the DB2 datasource.
# If the entry already exists, the function simply returns.
createJAAS(db2Alias, db2UserId, db2Password)
# Create the DB2 Universal Driver
print "Creating the DB2 Universal Driver Provider"
serverId = getServerId(nodeName,serverName)
jdbcProvider = createJdbcProvider ( serverId, "DB2 Universal JDBC Driver Provider", "${DB2UNIVERSAL_JDBC_DRIVER_PATH}/db2jcc.jar;${UNIVERSAL_JDBC_DRIVER_PATH}/db2cc_license_cu.jar;${DB2UNIVERSAL_JDBC_DRIVER_PATH}/db2jcc_license_cisuz.jar", "", "com.ibm.db2.jcc.DB2ConnectionPoolDataSource", "Non-XA DB2 Universal JDBC Driver-compliant Provider. Datasources created under this provider support only 1-phase commit processing except in the case where driver type 2 is used under WAS z/OS. On WAS z/OS, driver type 2 uses RRS and supports 2-phase commit processing." )
print "Created: jdbcProvider = " + jdbcProvider
# Create the JNDI Data Sources
# First, jdbc/users
print "Creating the Users Database Data Source"
usersDataSource = createDataSource( jdbcProvider, "Users Database", "jdbc/users", "jdbc/users", 10, db2Alias, "com.ibm.websphere.rsadapter.DB2UniversalDataStoreHelper" )
print "Created: usersDataSource = " + usersDataSource
setJ2eeResourceProperty( usersDataSource, "databaseName", "java.lang.String", db2DatabaseName, "The name of the database." )
setJ2eeResourceProperty( usersDataSource, "driverType", "java.lang.Integer", db2DriverType, "The JDBC connectivity-type of a data source." )
setJ2eeResourceProperty( usersDataSource, "serverName", "java.lang.String", db2ServerName, "The TCP/IP address or host name for the DRDA server." )
setJ2eeResourceProperty( usersDataSource, "portNumber", "java.lang.Integer", db2PortNumber, "The TCP/IP port number where the DRDA server resides." )
# Then, jdbc/archiva
print "Creating the Archiva Database Data Source"
archivaDataSource = createDataSource( jdbcProvider, "Archiva Database", "jdbc/archiva", "jdbc/archiva", 10, db2Alias, "com.ibm.websphere.rsadapter.DB2UniversalDataStoreHelper" )
print "Created: archivaDataSource = " + archivaDataSource
setJ2eeResourceProperty( archivaDataSource, "databaseName", "java.lang.String", db2DatabaseName, "The name of the database." )
setJ2eeResourceProperty( archivaDataSource, "driverType", "java.lang.Integer", db2DriverType, "The JDBC connectivity-type of a data source." )
setJ2eeResourceProperty( archivaDataSource, "serverName", "java.lang.String", db2ServerName, "The TCP/IP address or host name for the DRDA server." )
setJ2eeResourceProperty( archivaDataSource, "portNumber", "java.lang.Integer", db2PortNumber, "The TCP/IP port number where the DRDA server resides." )
See: https://cwiki.apache.org/ARCHIVA/archiva-on-websphere-61.html
Specifically, the file, createArchivaServer.py, from: https://cwiki.apache.org/ARCHIVA/archiva-on-websphere-61.data/createArchivaServer.py
And this is the relevant sections from the file:
# Specify the DB2 details.
db2DriverPath = "/usr/opt/db2_08_01/java"
db2Alias = "DB2ArchivaUserId"
db2UserId = "db2inst1"
db2Password = "db2inst1"
# JNDI details
db2DatabaseName = "ARCHIVA"
db2DriverType = "4"
db2ServerName = "localhost"
db2PortNumber = "50000"
and
# Set the WebSpere Variable needed to for the universal driver. In this case, we just set it for
# the individual server. To set it for the entire node, just remove the serverName variable completely.
print "Setting WebSphere Variable"
setWebSphereVariable( "DB2UNIVERSAL_JDBC_DRIVER_PATH", db2DriverPath, nodeName, serverName)
# Create a JAAS entry for the userid/password for the DB2 datasource.
# If the entry already exists, the function simply returns.
createJAAS(db2Alias, db2UserId, db2Password)
# Create the DB2 Universal Driver
print "Creating the DB2 Universal Driver Provider"
serverId = getServerId(nodeName,serverName)
jdbcProvider = createJdbcProvider ( serverId, "DB2 Universal JDBC Driver Provider", "${DB2UNIVERSAL_JDBC_DRIVER_PATH}/db2jcc.jar;${UNIVERSAL_JDBC_DRIVER_PATH}/db2cc_license_cu.jar;${DB2UNIVERSAL_JDBC_DRIVER_PATH}/db2jcc_license_cisuz.jar", "", "com.ibm.db2.jcc.DB2ConnectionPoolDataSource", "Non-XA DB2 Universal JDBC Driver-compliant Provider. Datasources created under this provider support only 1-phase commit processing except in the case where driver type 2 is used under WAS z/OS. On WAS z/OS, driver type 2 uses RRS and supports 2-phase commit processing." )
print "Created: jdbcProvider = " + jdbcProvider
# Create the JNDI Data Sources
# First, jdbc/users
print "Creating the Users Database Data Source"
usersDataSource = createDataSource( jdbcProvider, "Users Database", "jdbc/users", "jdbc/users", 10, db2Alias, "com.ibm.websphere.rsadapter.DB2UniversalDataStoreHelper" )
print "Created: usersDataSource = " + usersDataSource
setJ2eeResourceProperty( usersDataSource, "databaseName", "java.lang.String", db2DatabaseName, "The name of the database." )
setJ2eeResourceProperty( usersDataSource, "driverType", "java.lang.Integer", db2DriverType, "The JDBC connectivity-type of a data source." )
setJ2eeResourceProperty( usersDataSource, "serverName", "java.lang.String", db2ServerName, "The TCP/IP address or host name for the DRDA server." )
setJ2eeResourceProperty( usersDataSource, "portNumber", "java.lang.Integer", db2PortNumber, "The TCP/IP port number where the DRDA server resides." )
# Then, jdbc/archiva
print "Creating the Archiva Database Data Source"
archivaDataSource = createDataSource( jdbcProvider, "Archiva Database", "jdbc/archiva", "jdbc/archiva", 10, db2Alias, "com.ibm.websphere.rsadapter.DB2UniversalDataStoreHelper" )
print "Created: archivaDataSource = " + archivaDataSource
setJ2eeResourceProperty( archivaDataSource, "databaseName", "java.lang.String", db2DatabaseName, "The name of the database." )
setJ2eeResourceProperty( archivaDataSource, "driverType", "java.lang.Integer", db2DriverType, "The JDBC connectivity-type of a data source." )
setJ2eeResourceProperty( archivaDataSource, "serverName", "java.lang.String", db2ServerName, "The TCP/IP address or host name for the DRDA server." )
setJ2eeResourceProperty( archivaDataSource, "portNumber", "java.lang.Integer", db2PortNumber, "The TCP/IP port number where the DRDA server resides." )
Courtesy IBMer Chris Graham from down under.
Labels: configuration
Tuesday, August 30, 2011
Non standard performance tuning for the WebSphere Application Server
Links for some non standard and standard performance tuning of the WebSphere Application Server
http://www.ibm.com/developerworks/websphere/techjournal/0909_blythe/0909_blythe.html
http://wasdynacache.blogspot.com/2011/08/tune-your-websphere-application-server.html
http://wasdynacache.blogspot.com/2011/05/websphere-application-server-top-10.html
http://wasdynacache.blogspot.com/2010/03/pimp-my-websphere-reduce-websphere.html
http://wasdynacache.blogspot.com/2011/04/websphere-application-server-walk-in.html
http://wasdynacache.blogspot.com/2011/05/websphere-application-server.html
http://webspherecommunity.blogspot.com/2011/05/tuning-for-idleness-service.html
For a paid offering ask your IBM services representative about the Performance Tuning Configurator (PTC) tool for improving and tuning WAS performance
Labels: performance
IBM Extensions to the Eclipse memory analyzer
The IBM extended version of Eclipse Memory Analyzer contains a number of plugins that help with determining the memory usage of WAS components like Dynacache, HTTPSession, WebContainer etc .. The IBM Extensions for Memory Analyzer currently offers extensions covering: Java SE Runtime, & WebSphere Application Server
Happy Debugging!
Snapshot of what this look like in the tool ...
 |
| IBM WebSphere Application Server Extensions to Eclipse Memory Analyzer |
 |
| IBM Java SE Runtime Extensions to Eclipse Memory Analyzer |
If you have a complete core dump this tool can also show you thread details, thread stacks, a classloader explorer etc ...
 |
| Java Thread exploration with Eclipse Memory Analyzer |
 |
| My version of the IBM extended Eclipse Memory Analyzer |
Happy Debugging!
Labels: debugging
Thursday, August 18, 2011
WebSphere Session Persistence Best Practices ... The 10/10 Rule
10 Seconds write frequency /10 KB session size
The most important factor in session replication performance is the write frequency setting.
This setting should be carefully considered to determine if the environment needs full reliability and failover support (End of Servlet service), or if a time based frequency is sufficient. In testing done for this study, a value larger than 10 seconds rarely provided much additional benefit. Therefore, as a best practice, if the time based frequency is chosen, we would recommend a value of 10 seconds.
The cost of replication or persistence increases as the size of the session object increases.
Therefore, as a best practice, we would recommend keeping session sizes smaller than 10 KB.
Database session persistence performs better than Memory- to- Memory session persistence. This is because in the Database persistence mode, the application servers are not having to act as both an application server servicing requests and a replication backup, as was the case in the Memory-to-Memory scenarios. For larger environments, Database persistence is almost always the better choice as it scales a lot better than Memory-to-Memory.
There is an extra cost associated with a database server, license, and database administrator. Most environments require database backups / mirrors. The performance of the database can also be a bottleneck as the network latency between database and app server, and the speed of the database machine and disk drives are all factors. From a cost perspective Memory-to-Memory session persistence will always beat database persistence.
See this Whitepaper WebSphere Application Server V8 Replication Performance for a complete discussion of this topic and DayTrader benchmark results.
The most important factor in session replication performance is the write frequency setting.
This setting should be carefully considered to determine if the environment needs full reliability and failover support (End of Servlet service), or if a time based frequency is sufficient. In testing done for this study, a value larger than 10 seconds rarely provided much additional benefit. Therefore, as a best practice, if the time based frequency is chosen, we would recommend a value of 10 seconds.
The cost of replication or persistence increases as the size of the session object increases.
Therefore, as a best practice, we would recommend keeping session sizes smaller than 10 KB.
Database session persistence performs better than Memory- to- Memory session persistence. This is because in the Database persistence mode, the application servers are not having to act as both an application server servicing requests and a replication backup, as was the case in the Memory-to-Memory scenarios. For larger environments, Database persistence is almost always the better choice as it scales a lot better than Memory-to-Memory.
There is an extra cost associated with a database server, license, and database administrator. Most environments require database backups / mirrors. The performance of the database can also be a bottleneck as the network latency between database and app server, and the speed of the database machine and disk drives are all factors. From a cost perspective Memory-to-Memory session persistence will always beat database persistence.
See this Whitepaper WebSphere Application Server V8 Replication Performance for a complete discussion of this topic and DayTrader benchmark results.
Labels: session
Wednesday, August 17, 2011
Diagnostic tools for WebSphere Application Server
There are a number of publicly available tools and scripts that WebSphere Application Server component and service teams use for assisting and problem determination of PMRs.
WebSphere Plugin tools - Perl Script for parsing WebSphere Plugin logs
https://github.com/covener/plugin-tools
HTTPSession - Session tracker debug Servlet
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=pix&product=was-express-dist&topic=rtrb_httpsesncomp
Servlet name: com.ibm.ws.webcontainer.httpsession.IBMTrackerDebug shipped with WAS.
Dynacache: Extended Cache Monitor and Flight Recorder
http://wasdynacache.blogspot.com/2010/03/dynacache-flight-recorder-wsadmin.html
http://www.ibm.com/developerworks/websphere/downloads/cache_monitor.html
HAManager: Awk scripts for analyzing HAManager logs
https://www.ibm.com/developerworks/mydeveloperworks/files/app?lang=en_US#/person/0600007J6A/file/2499709b-2511-44ac-a38a-fa62b3e3e7aa
SIBus –Explorer & Destination Handler
Sun JVM ONLY Tools- jps, jstatd, jstack, jmap, jinfo, jhat visualVM & jConsole
Name space dumper $WAS_HOME/bin/dumpNameSpace tool
OSGI Bundle Explorer and debug utility $WAS_HOME/bin/osgiConsole.sh/bat
WebSphere Plugin tools - Perl Script for parsing WebSphere Plugin logs
https://github.com/covener/plugin-tools
HTTPSession - Session tracker debug Servlet
http://www14.software.ibm.com/webapp/wsbroker/redirect?version=pix&product=was-express-dist&topic=rtrb_httpsesncomp
Servlet name: com.ibm.ws.webcontainer.httpsession.IBMTrackerDebug shipped with WAS.
Dynacache: Extended Cache Monitor and Flight Recorder
http://wasdynacache.blogspot.com/2010/03/dynacache-flight-recorder-wsadmin.html
http://www.ibm.com/developerworks/websphere/downloads/cache_monitor.html
HAManager: Awk scripts for analyzing HAManager logs
https://www.ibm.com/developerworks/mydeveloperworks/files/app?lang=en_US#/person/0600007J6A/file/2499709b-2511-44ac-a38a-fa62b3e3e7aa
SIBus –Explorer & Destination Handler
- SIB Explorer: http://alphaworks.ibm.com/tech/sibexplorer
- SIB Performance Tool: http://www.alphaworks.ibm.com/tech/sibperf
- A tool to view, move, copy, delete, save and restore messages in the default messaging provider: http://www-01.ibm.com/support/docview.wss?uid=swg24021439
LDAP Tools
Apache Directory Studio Very handy LDAP tool based on Eclipse http://directory.apache.org/studio/index.html
ldapSearch - LDAP search tool http://linux.die.net/man/1/ldapsearch
Sun JVM ONLY Tools- jps, jstatd, jstack, jmap, jinfo, jhat visualVM & jConsole
OSGI Bundle Explorer and debug utility $WAS_HOME/bin/osgiConsole.sh/bat
ISA Tools
Labels: debugging tools
Thursday, August 11, 2011
Generating system dumps and thread dumps automatically
Generating a system dump instead of a heap dump automatically on an OOM
To generate a system dump as default on an OOM error or when the JVM receives the SIGQUIT.
Please set the following JVM generic custom property
-Xdump:java+system:events=systhrow,filter=java/lang/OutOfMemoryError,range=1..4,priority=999,request=exclusive+compact+prepwalk
Please also ensure that there is adequate space for the system dump by following this technote .
Generating a thread dump automatically on a thread hang
To configure a hang detection policy that dumps a javacore when a thread is hung by setting the following property
- From the administrative console, click Servers > Application Servers > server_name OR System Administration > Deployment Manager or System Administration > NodeAgent > node_agent_name
- Under Server Infrastructure, click Administration > Custom Properties
- Click New.
- Add the following property:
Value true
This causes a javacore to be created when a hung thread is detected and a WSVR0605W message is printed. The threads section of the javacore can be analyzed to determine what the reported thread and other related threads are doing.
Table of where to set a JVM Custom property
| Application server | Servers > Server Types > WebSphere application servers > server_name > Java and Process Management > Process Definition > Java Virtual Machine > Custom Properties |
| Deployment manager | System Administration > Deployment Manager > Java and Process Management > Process definition > > Java Virtual Machine > Custom Properties |
| Node agent | System Administration >Node Agent > nodeagent > Java and Process Management > Process definition > Java Virtual Machine > Custom Properties |
References:
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.diagnostics.60%2Fdiag%2Ftools%2Fdiag_collector.html
http:http//publib.boulder.ibm.com/infocenter/javasdk v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.diagnostics.60%2Fdiag%2Ftools%2Fdiag_collector.html
Tuesday, August 9, 2011
JAX-RS implementations in WebSphere Application Server
Jersey is the open source JAX-RS (JSR 311) Reference Implementation for building RESTful Web services. Jersey provides an API so that developers may extend Jersey to suit their needs. See blog post on how to integrate Jersey with WebSphere Application Server 6.1. If the first suggestion does not work look at this post.
Gotcha To get Jersey working with WAS v7 you will need to set the webcontainer custom property.
Name: com.ibm.ws.webcontainer.invokeFiltersCompatibility
Value: true
This workaround is NOT needed in WAS v8. See APAR PK56247 for details on why you need to do this in WAS v7.
If your application that leverages Jersey throws the following error during deployment
The URI scheme wsjar of the URI wsjar:file:/C:/rad802/workspace/myCompany/TestJerseyProvider/WebContent/WEB-INF/lib/Providers.jar!/com/mycompany/commons/web/rest/providers is not supported. Package scanning deployment is not supported for such URIs.
This occurs because your classes in web-inf/lib are not being recognized, because the Jersey URI Scheme scanner which scans WEB-INF/lib only works, when the protocol has 3 digits (like jar:) but WebSphere uses wsjar: as protocol.
We ran into a similar issue when using Open Web Beans see https://issues.apache.org/jira/browse/OWB-577
You can also look at the svn change history to see how we fixed it in Open Web Beans.
This is fixable in Jersey by adding your own com.sun.jersey.core.spi.scanning.uri.UriSchemeScanner which is the interface for scanning URI-based resources and reporting those resources to a scanning listener. Jersey's code add's additional UriSchemeScanner's using the com.sun.jersey.spi.service.ServiceFinder. Looking at existing scanners like the the com.sun.jersey.core.spi.scanning.uri.JarZipSchemeScanner & com.sun.jersey.core.spi.scanning.uri.FileSchemeScanner you will be able to write a scanner that is able to scan the jars in the WEB-INF/lib directory of your WAS application.
Another option is to do what the warning says Try using a different deployment mechanism such as explicitly declaring root resource and provider classes using an extension of javax.ws.rs.core.Application.
Please note that WAS ships with its own JAX-RS implementation based off the Apache Wink Project. This is available by default in WAS version 8 and with the Web 2.0 feature pack in WAS v7.
Labels: JEE
Saturday, August 6, 2011
Tune your WebSphere Application Server deployment for Production or Development
Do you know that WebSphere Application Server install contains scripts to tune the performance of the server ? These can be found in $WAS_HOME/scriptLibraries/perfTuning/V70
This wsadmin script can be run as follows ....
$WAS_HOME/bin/wsadmin -f applyPerfTuningTemplate.py [-nodeName <node> -serverName <server>] [-clusterName <cluster>] -templateFile <file> where templateFile could be one of "peak.props", "development.props" or "default.props"
#-----------------------------------------------------------------------------
# File Name:# ApplyPerfTuning.py
# Purpose:
# Apply pre-defined performance tuning templates to a specific
# application server instance or all instances in a cluster. Three
# templates are pre-defined (default, peak, and development);
# however, these files can easily be modified to create your own custom
# tuning template.
#-----------------------------------------------------------------------------
You can tune WAS either for development (faster application restarts, smaller footprint, dynamic threadpool sizes) or production (increased throughput, smaller garbage collection pauses, Fixed threadpool sizes). Correctly tuning WAS can vastly improve your developer and customer experience.
For a complete case study in basic and advanced tuning see http://www.ibm.com/developerworks/websphere/techjournal/0909_blythe/0909_blythe.html . The article discussed this core set of tunables plus other option resulting in performance gains in excess of 100%.
Labels: performance
Friday, August 5, 2011
Configure WebSphere Application Server Dynacache and Session persistence to use WebSphere eXtreme Scale
Dynacache
Please see instructions below on how to configure Dynacache to leverage WXS instead of the default caching engine. I have also provided links for detailed instructions if you get confused or stuck/
Video eXtreme Scale Wiki Configuring cache integration
Redbook chapter on integrating WXS with Dynacache in the context of WebSphere Commerce
You will also be interested in configuring session persistence with WXS.
Infocenter Configuring the HTTP session manager with WebSphere Application Server
Attention: With Version 7.1.0.3 and later, you can persist sessions that use URL rewriting or cookies as a session tracking mechanism. Before Version 7.1.0.3, you can save only sessions that use cookies as the session tracking mechanism to the data grid.
Cheatsheet
1. Install & Set up an WXS 7 grid (Catalog server and other XS containers that host the cache) . The latest WXS version has an admin console for monitoring. Please install the latest WXS 7 fix pack before proceeding.
2. Augment the WAS profile with the WXS client
3. Configure the dmap to use extreme scale
Basically add the following properties to the map where we create the dmap
com.ibm.ws.cache.CacheConfig.enableCacheReplication = true
com.ibm.ws.cache.CacheConfig.cacheProviderName = com.ibm.ws.objectgrid.dynacache.CacheProviderImpl
com.ibm.websphere.xs.dynacache.topology = remote
com.ibm.websphere.xs.dynacache.num_initial_containers = 1
4. [optional] XS does not place replica and primary shards on containers that have the same IP address (unless the configuration is in development mode). If ALL your XS containers are on one physical machine or node you will need to set development=true. In the deployment policy descriptor XML file after WXS client is installed i.e. $WAS-HOME/optionalLibraries/ObjectGrid/dynacache/etc/dynacache-remote-deployment.xml set developmentMode to true
<mapSet name="DYNACACHE_REMOTE"
numberOfPartitions="103"
minSyncReplicas="0"
maxSyncReplicas="0"
maxAsyncReplicas="1"
numInitialContainers="1"
developmentMode="false" --> true
replicaReadEnabled="false">
<map ref="IBM_DC_PARTITIONED_.*" />
<mapSet name="DYNACACHE_REMOTE"
numberOfPartitions="103"
minSyncReplicas="0"
maxSyncReplicas="0"
maxAsyncReplicas="1"
numInitialContainers="1"
developmentMode="false" --> true
replicaReadEnabled="false">
<map ref="IBM_DC_PARTITIONED_.*" />
Labels: caching
Wednesday, August 3, 2011
WebSphere Application Server Defibrillator for stalled software
Food for thought ...
I have a lot of interest in Java serviceability and JVM egregious condition recovery. If you share the same interest, please check out the following research paper http://groups.csail.mit.edu/pac/jolt/paper.pdf that talks about Detecting and Escaping In nite Loops with Java or the more easily digestible http://web.mit.edu/newsoffice/2011/infinite-loops-interrupt-0802.html
One thought that did resonate in the article is from Westley Weimer, an assistant professor of computer science at the University of Virginia who says "The vast majority of software engineering or programming-language researchers are shackled to this notion of full correctness or full soundness: You can’t change anything in the program if there’s even the possibility of getting a slightly wrong answer,”
The article goes on to say that “One of the things that’s really characterized Martin’s research over the last 10 or 15 years is casting off the shackles of soundness in favor of approaches that are probably correct but dramatically more useful in real life.”
This is true in order to achieve resiliency you will need to sacrifice some program consistency and correctness.
I have a lot of interest in Java serviceability and JVM egregious condition recovery. If you share the same interest, please check out the following research paper http://groups.csail.mit.edu/pac/jolt/paper.pdf that talks about Detecting and Escaping In nite Loops with Java or the more easily digestible http://web.mit.edu/newsoffice/2011/infinite-loops-interrupt-0802.html
One thought that did resonate in the article is from Westley Weimer, an assistant professor of computer science at the University of Virginia who says "The vast majority of software engineering or programming-language researchers are shackled to this notion of full correctness or full soundness: You can’t change anything in the program if there’s even the possibility of getting a slightly wrong answer,”
The article goes on to say that “One of the things that’s really characterized Martin’s research over the last 10 or 15 years is casting off the shackles of soundness in favor of approaches that are probably correct but dramatically more useful in real life.”
This is true in order to achieve resiliency you will need to sacrifice some program consistency and correctness.
WebSphere Application Server SWAT BLOG
For all those interested in WebSphere Application Server serviceability and problem determination please follow the WAS Swat blog run by Kevin Grigorenko WAS troubleshooting extraordinaire https://www.ibm.com/developerworks/mydeveloperworks/blogs/kevgrig/?lang=en
Tuesday, August 2, 2011
How to analyze IBM JVM core dumps
Today's blog post is advice, I provided to a customer for analyzing IBM JDK core dumps
First see these instructions to run the dump analyzer outside the ISA
cd $ISA_HOME\applications\eclipse\plugins\com.ibm.java.diagnostics.dbda_2.2.2.20090926232659
java -Xmx1400M -cp dumpAnalyzer.jar;lib\dtfj.jar;..\com.ibm.java.diagnostics.dbda.was_2.1.3.20090427213613\wasAnalyzers.jar;..\com.ibm.java.diagnostics.dbda.core_2.2.2.20090926232659\coreAnalyzers.jar com.ibm.dtfj.analyzer.base.DumpAnalyzer core1.sdff com.ibm.dtfj.analyzer.jvm.DefaultDumpReport_Extended > DumpAnalyzer_DefaultDumpReport_Extended.txt
OR
You basically need to copy the 6 required jars from IBM Support Assistant and then run the command
set CP="dumpAnalyzer.jar;coreAnalyzers.jar"
set BCP="lib/j9/dtfj.jar;lib/j9/dtfj-interface.jar;lib/sov/dtfjsu.jar;lib/sov/dtfj.jar"
java -cp $CP -Xbootclasspath/p:$BCP
com.ibm.dtfj.analyzer.base.DumpAnalyzer (dumpName) (options)
You need to put this in a script and run it whenever a core dump is generated.
Happy Debugging!
Question ...
We have a problem where our Linux-based build machines for the portal drivers, occasionally core dump in the middle of the build process. The only way I know of to invoke the Java team to shoot these, is to open a WAS PMR. I'm not sure that's appropriate, given that it's unclear exactly what JRE is running at the time (it could be the one from the WAS or from ant)
Response
In order to analyze a coredump you will need to run the Dump Analyzer plugin from IBM Support Assistant (ISA). The dump report will tell you the details of the JVM that dumped core, the heap contents and the javacore. This will help you determine the culprit component faster.First see these instructions to run the dump analyzer outside the ISA
cd $ISA_HOME\applications\eclipse\plugins\com.ibm.java.diagnostics.dbda_2.2.2.20090926232659
java -Xmx1400M -cp dumpAnalyzer.jar;lib\dtfj.jar;..\com.ibm.java.diagnostics.dbda.was_2.1.3.20090427213613\wasAnalyzers.jar;..\com.ibm.java.diagnostics.dbda.core_2.2.2.20090926232659\coreAnalyzers.jar com.ibm.dtfj.analyzer.base.DumpAnalyzer core1.sdff com.ibm.dtfj.analyzer.jvm.DefaultDumpReport_Extended > DumpAnalyzer_DefaultDumpReport_Extended.txt
OR
You basically need to copy the 6 required jars from IBM Support Assistant and then run the command
set CP="dumpAnalyzer.jar;coreAnalyzers.jar"
set BCP="lib/j9/dtfj.jar;lib/j9/dtfj-interface.jar;lib/sov/dtfjsu.jar;lib/sov/dtfj.jar"
java -cp $CP -Xbootclasspath/p:$BCP
com.ibm.dtfj.analyzer.base.DumpAnalyzer (dumpName) (options)
You need to put this in a script and run it whenever a core dump is generated.
Happy Debugging!
Labels: debugging
Monday, August 1, 2011
Solving an application performance slowdown or regression with WebSphere Application Server
Recently I had the chance to troubleshoot and fix a performance regression a component of the WebSphere Application Server. The objective was to determine performance and locking hotspots in our server code.
1. Beg, borrow steal or write a benchmark. We already have a pretty good Dynacache performance benchmark. This is critical to draw a line in the sand and determine a before and after performance #s.
2. Enable IBM Health Center profiling. Using IBM Health Center which is baked into the WAS JVM, is the easiest and lightest way of doing profiling with WebSphere Application Server. No expensive profiling tools and no instrumentation of src. This is best thing since baked bread for performance in WAS.
3. So you need two things for this exercise. On the client side you need an IBM Health Center Client that comes as a free product plugin to the IBM Support Assistant. On the server side you need IBM Health Center server library which comes preinstalled with the JVM which is bundled with WAS. You will find one WAS_HOME/java/jre/lib/ext/healthcenter.jar
4. Configure WAS to use the IBM Health Center. See details in Infocenter for older versions of java.
Servers ->Server Types -> WebSphere application servers. Select the server name and then select Java and Process Management -> Process definition -> Java Virtual Machine -> Generic JVM Arguments.
For Java 5 SR10 and later, or Java 6 SR5 and later, use:-Xhealthcenter
5. After restaring WAS, start the ISA and configure a connection to your WAS server. This connection is established typically on port 1872. If you want to control the server port see infocenter. After the JVM is started with the agent enabled, you see a message detailing the port for the Health Center agent. For example:
05-Mar-2009 09:49:57 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgentINFO: Health Center agent started on port 1972.
6. Run the benchmark or the test.
7. After the test save the profile as a *.hcd file. You can then pass this profile around and analyze it offline. See technote.
8. Interpret the method profile. The Method Profile table shows which methods are using the most processing resource.
Happy profiling!
1. Beg, borrow steal or write a benchmark. We already have a pretty good Dynacache performance benchmark. This is critical to draw a line in the sand and determine a before and after performance #s.
2. Enable IBM Health Center profiling. Using IBM Health Center which is baked into the WAS JVM, is the easiest and lightest way of doing profiling with WebSphere Application Server. No expensive profiling tools and no instrumentation of src. This is best thing since baked bread for performance in WAS.
3. So you need two things for this exercise. On the client side you need an IBM Health Center Client that comes as a free product plugin to the IBM Support Assistant. On the server side you need IBM Health Center server library which comes preinstalled with the JVM which is bundled with WAS. You will find one WAS_HOME/java/jre/lib/ext/healthcenter.jar
4. Configure WAS to use the IBM Health Center. See details in Infocenter for older versions of java.
Servers ->Server Types -> WebSphere application servers. Select the server name and then select Java and Process Management -> Process definition -> Java Virtual Machine -> Generic JVM Arguments.
For Java 5 SR10 and later, or Java 6 SR5 and later, use:-Xhealthcenter
5. After restaring WAS, start the ISA and configure a connection to your WAS server. This connection is established typically on port 1872. If you want to control the server port see infocenter. After the JVM is started with the agent enabled, you see a message detailing the port for the Health Center agent. For example:
05-Mar-2009 09:49:57 com.ibm.java.diagnostics.healthcenter.agent.mbean.HCLaunchMBean startAgentINFO: Health Center agent started on port 1972.
6. Run the benchmark or the test.
7. After the test save the profile as a *.hcd file. You can then pass this profile around and analyze it offline. See technote.
8. Interpret the method profile. The Method Profile table shows which methods are using the most processing resource.
- Methods with a higher Self (%) value are described as "hot", and are good candidates for optimization. Small improvements to the efficiency of these methods might have a large effect on performance. Methods near the bottom of the table are poor candidates for optimization. Even large improvements to their efficiency are unlikely to affect performance, because they do not use as much processing resource.
- You can optimize methods by reducing the amount of work that they do or by reducing the number of times that they are called. Highlighting a method in the table populates the call hierarchy views.Filter the contents of the method profile table using the text box above the table. See the filtering help topic for more information.
- Additionally, when you select the Hide low sample entries, the table does not list any entries that have a sample count of less than 2. Use this option if your table contains many entries that are not obvious candidates for optimization to improve the performance of the table.
9. Make java source code changes to improve performance. Scrub and repeat. We used the process above to take before and after profiles to figure out the delta i.e .where the performance had slipped.
Happy profiling!
Labels: application performance
Archives
December 2006 September 2008 January 2009 February 2009 March 2009 September 2009 October 2009 November 2009 December 2009 January 2010 February 2010 March 2010 April 2010 October 2010 January 2011 February 2011 April 2011 May 2011 June 2011 July 2011 August 2011 September 2011 October 2011 November 2011 December 2011 January 2012 February 2012 March 2012 April 2012 May 2012 June 2012 July 2012 August 2012 September 2012 October 2012 November 2012 January 2013 May 2013 June 2013 July 2013 September 2013 October 2013 June 2014 August 2014
Subscribe to Posts [Atom]