Remote debugging with Eclispe for Applications deployed on Pramati Server
The Java Debugger (jdb) is a dynamic, controlled, assignment-based debugging tool. It helps to find and fix bugs in the Java language programs both locally and on the server. To use jdb with Pramati Server, follow the provided steps below:
1. Edit runserver.bat/runserver.sh file located at <SERVER_INSTALL>\server\bin directory.
2. Add the following properties:
-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n
Eg:
set TEMP1=%PATH%
set DEBUG_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n
call E:\Servers\PServer50\SP3_test\server\bin\setup.bat
java -server %DEBUG_OPTS% -Xms128m -Xmx512m -Dinstall.root=%install_root% -verbose:gc -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -Xloggc:verbose_gc.log -Dinstall.root=%install_root% -Djava.security.policy=%install_root%\lib\pramati\pramati.java.policy -Djava.security.auth.policy=%install_root%\lib\pramati\pramati.jaas.policy -Djacorb.home=%install_root%\lib\tp\jacorb -Djava.endorsed.dirs=%install_root%\lib\std\endorsed com.pramati.Server %*
@ set PATH=%TEMP1%
3. Make sure that the address port (used in above configuration) should be free.
4. Start Pramati Server with the above mentioned parameters.
5. Once Pramati Server has started successfully, start Eclipse.
6. Now, in Eclipse, navigate to Run | Debug option.
7. This will open a dubug window. In this window, right click on Remote Java Application and select New option
8. In the Create configuration screen you’ll be prompted to enter the following values:
* Name : Name of the Debugger name, can be any name
* Project: Select the Java project that contains the source code you want to debug.
* Connection Type: Leave Connection Type in default, i.e. Standard (Socket Attach) .
* Host: enter localhost. If you want to debug a remote server, enter it’s hostname or IP address.
* Port: enter 8000 (as defined in your Pramati Server’s startup script).
9. Click Apply
10. In the same screen click Debug (Make sure Pramati Server instance is already running in debug mode with suggested prameters). Eclipse should automatically take you to the Debug perspective and you should see a stack trace in the Debug view.
11. If you are not automatically taken to the Debug perspective, select Window | Open Perspective | Other and then click Debug option.
12. Now, add the break points and make a request for the application using browser.
13. The application shall be executed till to reach break point. Once break point has reached, Eclipse Debug window should automatically pop-up with the stack pointer on your first breakpoint. This confirms that the whole debug set up is working fine.
Collect thread dumps while running java process in background
Thread dumps are vital information to debug the application response slow issue and know the server status. If Pramati Server is running as a background process (Service) in Windows, it is very difficult to collect thread dumps. To overcome this problem, use one of the following third party tools:
1. StackTrace tool developed by adaptj
2. SendSignal program
1. StackTrace is a tool developed by adaptj for inspecting the state of a running JVM without starting it in debug mode and without using any additional startup settings. Visit the following link to download the same:
http://www.adaptj.com/root/main/download
* Now, click on “Launch” button and save the “launch.jnlp” file in the local system.
* Once the launch.jnlp file has been downloaded successfully, double click on the lanuch.jnlp file to execute. Please note that this jnlp file requires Internet connection during the execution.
* Upon the startup, it opens a new window with the title of “Stack Trace-Untitled”.
* In the opened window, provide “Process ID” (Pramati Server’s java process id located in task manager) in the textbox.
* Now, select “Process” tab and then choose “Thread dump” option.
* Upon the selection, it will generates the thread dumps in that tool itself.
2. SendSignal is a tool which sends ctrl-break signal from command line to JVM to generate thread dumps. Visit the following link to download the same:
http://www.latenighthacking.com/projects/2003/sendSignal/SendSignal.exe
* Once you have downloaded SendSignal.exe program from the above mentioned link, open a command prompt and navigate to the directory where you have downloaded this program.
* Increase the screen buffer(right click on the left top corner of the command window–>Properties–>Layout–>Increase the screen buffer height to 9999) size to capture complete thread dumps.
* Now, Execute the following command:
SendSignal <$PID>
Where PID is the java process id related to Pramati Server. We will get this from task manager.
* This command will generate the thread dumps on the command prompt.
Enforcing SSL Protocol and Minimum Encryption Key Strength with Pramati Server
While taking advantage of secured communication using https protocol, some times it is desirable to customize the default behavior of the ssl negotiation. This could be enforcing use of a more secured protocol than outdated ones or enforcing use of a minimum encryption key strength.
Depending on which protocol is used and the encryption key strength, both browser(client) and server maintain a list of supported Cipher Suites. User can choose which Cipher Suites to allow by restricting the server supported Cipher Suites. These server side Cipher Suites can be chosen depending on minimum encryption key strength desired .
Pramati server now comes with inbuilt support for such customizations which can be enabled or disabled using certain start-up options. The feature is available on Pramati Server Versions 5.0SP4 onwards.
The following JVM options are available for enabling various protocols and cipher suites:
i) com.pramati.web.ssl.enabledProtocols (Enabled SSL Protocols comma separated values)
ii) com.pramati.web.ssl.enabledCipherSuites (Enabled Cipher Suites comma separated values)
iii) com.pramati.web.ssl.disabledCipherSuites (Disabled Cipher Suites comma separated values)
Using above options, the enabled protocols can be specified and the undesired cipher suites with low bit encryption can be disabled as follows:
-Dcom.pramati.web.ssl.enabledProtocols=SSLv3,TLSv1
-Dcom.pramati.web.ssl.disabledCipherSuites=EXP1024-DHE-DSS-DES-CBC-SHA,EXP1024-DES-CBC-SHA,EXP1024-RC2-CBC-MD5,EDH-RSA-DES-CBC-SHA,EDH-DSS-DES-CBC-SHA,DES-CBC-SHA,EXP1024-DHE-DSS-RC4-SHA,EXP1024-RC4-SHA,EXP1024-RC4-MD5,EXP-EDH-RSA-DES-CBC-SHA,EXP-EDH-DSS-DES-CBC-SHA,EXP-DES-CBC-SHA,EXP-RC2-CBC-MD5,EXP-RC4-MD5,EXP1024-DHE-DSS-DES-CBC-SHA,EXP1024-DES-CBC-SHA,EXP1024-RC2-CBC-MD5,EDH-RSA-DES-CBC-SHA,EDH-DSS-DES-CBC-SHA,DES-CBC-SHA,EXP1024-DHE-DSS-RC4-SHA,EXP1024-RC4-SHA,EXP1024-RC4-MD5,EXP-EDH-RSA-DES-CBC-SHA,EXP-EDH-DSS-DES-CBC-SHA,EXP-DES-CBC-SHA,EXP-RC2-CBC-MD5,EXP-RC4-MD5
The above properties need to added into the <server-install>/server/bin/runserver.bat (assuming that this batch file is used to start the server)as following:
java -server -Dcom.pramati.web.ssl.enabledProtocols=SSLv3,TLSv1 -Dcom.pramati.web.ssl.disabledCipherSuites=EXP1024-DHE-DSS-DES-CBC-SHA,EXP1024-DES-CBC-SHA,EXP1024-RC2-CBC-MD5,EDH-RSA-DES-CBC-SHA,EDH-DSS-DES-CBC-SHA,DES-CBC-SHA,EXP1024-DHE-DSS-RC4-SHA,EXP1024-RC4-SHA,EXP1024-RC4-MD5,EXP-EDH-RSA-DES-CBC-SHA,EXP-EDH-DSS-DES-CBC-SHA,EXP-DES-CBC-SHA,EXP-RC2-CBC-MD5,EXP-RC4-MD5,EXP1024-DHE-DSS-DES-CBC-SHA,EXP1024-DES-CBC-SHA,EXP1024-RC2-CBC-MD5,EDH-RSA-DES-CBC-SHA,EDH-DSS-DES-CBC-SHA,DES-CBC-SHA,EXP1024-DHE-DSS-RC4-SHA,EXP1024-RC4-SHA,EXP1024-RC4-MD5,EXP-EDH-RSA-DES-CBC-SHA,EXP-EDH-DSS-DES-CBC-SHA,EXP-DES-CBC-SHA,EXP-RC2-CBC-MD5,EXP-RC4-MD5 -Dinstall.root=$install_root -Djava.security.policy=$install_root/lib/pramati/pramati.java.policy -Djava.security.auth.policy=$install_root/lib/pramati/pramati.jaas.policy -Djacorb.home=$install_root/lib/tp/jacorb -Djava.endorsed.dirs=$install_root/lib/std/endorsed com.pramati.Server
For *nix and Solaris systems the runserver.sh file can be used. In case the server is started as service, these options can be listed in the JVM options section in <server-install>/services/j2eeserver/pramati_server_<node-name>.conf
wrapper.java.additional.<n>=-Dcom.pramati.web.ssl.enabledProtocols=SSL……
wrapper.java.additional.<n+1>=--Dcom.pramati.web.ssl.disabledCipherSuites=……
<n> being the number next to the already existing parameter list in the “wrapper.java.additional” section.
The Cipher suites mentioned here are for example only. User can replace them with the names of applicable cipher suites that is intended to blocked. If only selected cipher suites need to be allowed user can use the “-Dcom.pramati.web.ssl.enabledCipherSuites=……..” option.
As these changes are passed on as JVM options, a restart of server shall be required to apply the changes.
Note: The SSL handshake is at Socket level which is handled by the underlying JDK. The above mentioned options are hence internally passed on to JVM for restricting the SSL communication. As such, without these options in place server shall support all Protocols and Ciphers as supported by the underlying JDK/JVM.
Apache mod_proxy and Pramati Server
Apache 2.0.44 and later versions have the built in redirector named “mod_proxy”, when configured, will redirect the request to the back-end Server. The back-end can be any Web Server, here we are talking specifically about Pramati Server. Apache server will be in front, taking all the HTTP request and will redirect these requests to Pramati Server based on the requested context. User will never feel that he got the response from Pramati Server, not from the Apache server. Lets us see how we can do this.
1) Stop the Apache server if running.
2) Navigate to the Apache installation, lets call it “APACHE_HOME”. For example, if your Apache is installed at “C:\Apache2.2″, then this is your “APACHE_HOME”
3) Locate the httpd.conf file under APACHE_HOME\conf directory and uncomment the following line:
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
4) Add the following to the httpd.conf file located at APACHE_HOME\conf:
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass /jspDeskWeb http://<host_name>:<port_name>/jspDeskWeb
ProxyPassReverse /jspDeskWeb http://<host_name>:<port_name>/jspDeskWeb
Where,
* “jspDeskWeb” : denotes the context root for which need the requests to be redirected to the back-end Pramati Server. You can provide your own application context root
* <host_name>:<port_name> : the host:http-port of Pramati Server on which the application is deployed.
5) You must replace the values of context root, IP and Port as per your Pramati Installation.
6) Start the Pramati Server now.
7) Save all the changes and restart Apache HTTP server. Now Apache is ready to redirect requests to Pramati Server which:
* Runs on host mentioned in ProxyPass URL
* Listens on http port number mentioned in ProxyPass URL
* And have the application deployed on it
Now Client will send the request as “http://<host_name_of_apache>/contextRoot/…..” and request will be served from Pramati Server.
Capturing Trace Logs on Pramati Server
Often in our application we need to analyse in detail the cause for a response (usually undesired) from a J2EE application server. Well, needless to say that the details can be best analysed by the Guys who implemented that server code, however such logs can definitely provide a vivid picture of the whole processing cycle.
On Pramati Server such a detailed log is called Trace Log. Best part being that the same can be enabled or disabled in runtime. The output can be printed to a console or saved to a file for future review. The trace options, like component for which trace is enabled, Logging severity level, file logging or console print etc., can be specified dynamically or can be read from a properties file.
A generic Trace Properteis file : Say server_trace.props: can contain:
handler.pramatihandler.contexttype.packages=.,true
handlers=pramatihandler
loglevel=debug
asynch=true
handler.pramatihandler.details=servertrace.log
handler.pramatihandler.type=file
This indicates that with these properties loaded, the trace logs shall be logged with minimum of Debug level, into a file named servertrace.log (in the logs directory of the node) for all packages (like web, ejb, jms ….).
With Pramati Server Running on a machine, you might most likely to have the j2eeadmin@<node-name> prompt available to you.
To Start a detailed logging of the server side activities, you can load trace by executing the following command at the above prompt. For example:
trace load server_trace.props
The server_trace.props file should be located in the <server-install>/server/bin directory in this case. If trace is loaded successfully, the following message should appear on the console
Successfully loaded trace properties
Now, with trace loaded we can carryout the task for which the detailed trace log is needed. This can involve in accessing a web page or invoking a java client etc. As we carry out the activity, servers activity logs shall be populated in the servertrace.log.* files in <server-install>/server/nodes/<node-name>/logs directory.
Once we are done with our issue replication, we can unload trace by executing the following command:
trace unload
A confirmation message shall appear on the console that trace has been unloaded successfully.
Successfully unloaded trace properties
Now the servertrace.log.* files can be collected and reviewed for analysis.
Interactive Loading of Trace:
Trace can also be loaded interactively by executing the following command:
trace load
and interactively the required options can be provided. For example:
j2eeadmin@default> – asynch [true] #true
j2eeadmin@default> – Logging Level [info] #debug
j2eeadmin@default> – Type of logging [console] #file
j2eeadmin@default> – Output File Name * #server_trace.log
j2eeadmin@default> – Target Packages [all] #all
j2eeadmin@default> – Save File Name #server_trace.props
Such logs can be unloaded by similar command (trace unload).
Using Remote Shell to Load Trace:
If server j2eeadmin@<node-name> prompt is not available, remote shell can be used to load and unload trace as well. Information on connecting to server using remote shell can be located at http://server.pramati.com/blog/2009/12/14/the-pramati-server-remote-shell/
Note: Just make sure that the server_trace.props file, if used for loading trace, should be placed in the <server-install>/server/bin directory of the server installation, from where the remoteshell script is being invoked.
To collect Trace for any Issue:
The steps to be followed are
[Load Trace]- [Replicate the issue]-[Unload Trace]-[Collect Trace logs]- [Analyse]
Do remember to unload trace, as the trace logs are populated very fast and can consume lot of system resoureces if left loaded for long time.
Pramati LoadBalancer Best Practices
While setting up Pramati Load Balancer, we must configure it properly so that the basic purpose of balancing the load will be served. This requires two files to be configured properly, those are:
1) web-lbconfig.xml file located at <PramatiServer_Home>/server/nodes/<ld_node_name>/config directory
2) web-config.xml file of each back-end node.
Provided below with the few things to be taken care:
1) “web-lbconfig.xml” file
- Provide the required node name and port name in the <nodes> tag. The node name has to be unique and not necessarily the actual name of the node.
Eg:
<nodes self-name=”null”>
<node name=”node1″ host=”192.168.1.1″ web-port=”8383″>
<naming-port>9393</naming-port>
<socket-pool enabled=”true” min=”20″ max=”50″ pool-idle-timeout-seconds=”100″ />
</node>
</nodes>
- Under the “<node-choosers>” tag, please search and remove the comment from the tag “<host>” and provide the following details:
– name of the host
– uri pattern for the application
– reference node name
Eg:
<host name=”localhost”>
<uri pattern=”.*jspDeskWeb(.*)?” node-ref=”node1″/>
</host>
- The pool-idle-time-out-seconds should be set to less than than the min of keep-alive-timeout, idle-time-out of back-end nodes. Let us assume that, if the pool-idle-time-out value is x Secs, the keep-alive-timeout for the back-end node could be set with following equation:
(x+20)*1000 milli seconds.
Configuring this will ensure that the socket connections between the Loadbalancer and the back-end nodes will not get timed out for a long period of time. There will not be any unnecessary socket creation.
- If more than one back-end nodes are to be configured then provide all the nodes’ name in a comma separated list in the following manner:
Eg:
<host name=”@”>
<uri pattern=”.*jspDeskWeb(.*)?” node-ref=”webnode1,webnode2,webnode3,webnode4″/>
</host>
- Define the fail-over nodes if you are having more than one back-end node. Change the <failover-node-choosers> tag as shown below:
Eg:
<failover-node-choosers>
<node-chooser name=”generic-fail-over-node-chooser” enabled=”true”>
<failover failed-node-ref=”node1″ standby-node-ref=”node2,node3,node4″/>
<failover failed-node-ref=”node2″ standby-node-ref=”node3,node4,node1″/>
<failover failed-node-ref=”node3″ standby-node-ref=”node4,node2,node1″/>
<failover failed-node-ref=”node4″ standby-node-ref=”node1,node2,node3″/>
</node-chooser>
</failover-node-choosers>
- In case you have configured multiple back-end nodes, then the maximum socket pool size of each node configured in web-lbconfig.xml must be equal or less than the keep-alive thread count of each back-end node (as mentioned in the web-config.xml).
- Further, the total worker thread count on LB node must be equal with the total number of threads of all the back-end nodes. Something like below:
Eg:
If, in the “LB” node’s web-lbconfig.xml, we have two nodes, such as:
<nodes self-name=”null”>
<node name=”webnode1″ host=”localhost” web-port=”18793″>
<naming-port>9393</naming-port>
<socket-pool enabled=”true” min=”20″ max=”50″ pool-idle-timeout-seconds=”100″ />
</node>
<node name=”webnode2″ host=”localhost” web-port=”18793″>
<naming-port>9393</naming-port>
<socket-pool enabled=”true” min=”20″ max=”50″ pool-idle-timeout-seconds=”100″ />
</node>
</nodes>
then, the appropriate numbers of LB worker thread count and back-end node count would be:
In web-config.xml of LB node
<worker-thread-count max-active=”-1″ max-keepAlive=”100″>125</worker-thread-count>
In web-config.xml of nodes “webnode1″ & “webnode2″ (each)
<worker-thread-count max-active=”-1″ max-keepAlive=”50″>75</worker-thread-count>
Now with these tunings LoadBalancer is ready to serve request. Let us tune the back-end nodes as well. So the file to be tweaked is “web-config.xml” of each back-end node.
2) “web-config.xml” file
- Change the value of keep-alive-timeout (millis) to a high value. Preferably to 2-3 minutes. This value decides a request can wait for 2-3minutes on a socket.
- Disable the Denial Of Service on all the back-end nodes by setting the <denial-of-service> tag value to false
Eg:
<denial-of-service enabled=”false”>
- The <worker-thread-count> count value must be greater than the “<max-keepAlive>” value. The <worker-thread-count> count is the number of threads that process requests.
- The “max-keepAlive” attribute is the most number of worker threads that can be in a keep-alive wait state. Setting the value lesser than that of worker-thread-count allows for some threads to be always free to serve newer connections.
With these, the whole setup is tunned well to serve the load.
Capturing http headers for a request-response cycle
Often it is required to know if a specific header or cookie is sent /received as part of request/response to check functionality of an application or to debug an issue. Few browser specific tools help us in recording http headers as the browsers talk to the servers.
For Firefox, the plug-in is called Live HTTP Headers(https://addons.mozilla.org/en-US/firefox/addon/3829). You can always locate a compatible version of the plugin for your Firefox Version. Install the same. and restart Firefox. Now click on Tools->Live HTTP headers. A new window will open up which will collect the Headers info as we browse.
For Internet Explorer, the plug-in is IE Http Header (http://www.blunck.se/iehttpheaders/ieHTTPHeadersSetup.exe). Install the same and restart IE. Now Click on View->Explorer Bar->ieHTTPHeadersVx.x. Bottom part of current window shall be reserved for recording the Header info.
Firefox plugin provides a well structured Headers info by segregating the each request-response cycle headers in a separate block which helps in debugging concurrent requests.
If the Http header windows are kept open, headers shall be populated in the Headers window as you browse. The output in the window is a plain text output that can be saved to a text file for reference and analysis.
The Pramati Server Remote Shell
Remote Shell Motivation
Depending upon the startup method used, user may or may not have a server admin prompt available to control Pramati Server. If server is started using startup scripts like runserver.bat (runserver.sh for UNIX and SOLARIS OS) then, at the end of server startup user is presented with a prompt similar to
- j2eeadmin@<node-name>
Several useful commands like list, deploy, trace load/unload, status all can be executed at this prompt. What if server is started in the back ground ? Like through admin service or as OS service. You can use remote shell in that case.
Remote shell allows one to connect to a running instance of server and provides prompt similar to j2eeadmin@<node-name> prompt where server can be controlled (remotely). Commands that can be executed at the j2eeadmin@<node-name> prompt can also be executed at the remote shell prompt.
How to connect:
To connect to a running instance of server using remote shell:
- Navigate to <server-install>/server/bin directory on a command propmt
- Execute remoteshell.bat (./remoteshell.sh for UNIX and SOLARIS)
- The script execution shall halt at a prompt
not_connected:>
- type “connect” (without quotes) and press enter to start the connection process.
- Enter the values of IP and naming (RMI) port for the server you intend to connect to and admin username/password when prompted for. A sample is given below
not_connected:> -ip [localhost]#192.168.1.2
not_connected:> -port [9191]#9191
not_connected:> -realm [system]#system
not_connected:> -user [root]#root
not_connected:> -password #
Attempting to connect to remote server instance…Please wait
Connected to [192.168.1.2:9191] as [root@system]
- Note that the password characters shall not be prompted on the screen as you type, for security reasons.
- You will be left at a prompt as given below (depending on the node -name)
j2eeadmin@default> [Remotely Connected]
- Now, at this prompt, you can execute commands like the usual j2eeadmin@<node-name> prompt.
In case you are trying to connect to a running node on the same machine, you can use a more user friendly command, after navigating to <server-install>/server/bin directory
remoteshell.bat -node <node-name>
(./remoteshell.sh -node <node-name> for UNIX and SOLARIS)
This command shall use the port numbers as configured in the <node-name> node’s server-config.xml file and connect to the running instance of node on these ports.
Once you are done, just execute the command “exit” without the quotes to come out of remote shell.
Uses of remote shell:
- Connect to a server running in the back ground on the same mcahine
- Connect to a remote server and remotely administer the same
- If the original j2eeadmin@<node-name> prompt becomes unresponsive you can use the remote shell prompt to collect desired debug info or to collect the status of server.
- You can also use it to shut down the server gracefully if original command prompt becomes unresponsive
- You can use remote shell scripting abilities to execute commands non interactively to complete any routine activities on server like application redeployment (Read further for details)
Scripting with remote shell:
Remote shell command can be used along with specific arguments to execute repetitive tasks like application redeployment on running instance of a server. After navigating to <server-install>/server/bin directory, one can execute:
remoteshell.bat -node <node-name> -cmd deploy <path-to-application-archive>
(./remoteshell.sh -node <node-name> -cmd deploy <path-to-application-archive> for UNIX and SOLARIS OS)
to deploy/redeploy the application the running <node-name> node. Due to its simplicity the same can be use in a batch or script file for repetitive invocation if desired. Commands like list, “status all” and shutdown too can be executed in this manner.
Shared Library in Tomcat6.0
While working on a issue, I found that Tomcat 6.0 does not have any directory where we can put third-party jars (like our “<$pserver>/server/lib/ext” directory). Couple of hits on google revealed that now it is configurable rather than putting the file physically inside $CATALINA_HOME. It is something like a pluggable entity, put the jars into a directory and just provide the aboslute path inside conf/catalina.properties file. Inside this file, change the following property:
shared.loader
Eg:
shared.loader=E:\\shared\\drivers\\mysql-connector-java-3.1.11-bin.jar
But these jar will be picked up dynamically. Rather, Tomcat 6.0 needs a restart after this. And Driver will be loaded.
Please note that, prior to this version of Tomcat, external/third party jar could be placed inside following locations:
1) ${catalina.base}/shared/lib/ – jar needs to be shared by all the deployed applications only(to share classes across all web applications)
2)${catalina.base}/common/lib/ – jar need to be shared by web server and applications.
Library Class Loader In Pramati
In this “Class Loader” game, we are also in the same path with our new feature named “Library Class Loader”. This has been introduced in Pramati Server 6.0 SP2 version onwards. It is again a pluggable entity. If you know the heirarchy of Pramati classloader(If not, here you go http://www.pramati.com/docstore/1230021/dg_classloaders.pdf), then the new guy comes right after “Pramati Ext ClassLoader”. You can place the external jar files anywhere w.r.t.:
* <$pserver>/server/lib>
* <$pserver>/server/nodes/<node_name>
* <$pserver>/server/nodes/<node_name>/config
This is just like a common library, where all the server integrated jar files are placed. If any application needs to use these jars then:
* configure path in <$pserver>/server/nodes/<node_name>/config/server-config.xml as :
<library-classloader>
<dir>$LIB_DIR/tp/spring/core</dir>
<classpath-entry>$LIB_DIR/pramati/pramati_spring_integration.jar</classpath-entry>
<prefer-library-classes>false</prefer-library-classes>
</library-classloader>
* use the tag “use-library-classes” and make the value as “true” in the Application’s pramati-j2ee-server.xml file
And you are ready to use the libraries jar file in respective applications.