Tuesday, March 18, 2014

RHQ: monitoring JMX beans of a Java EE application deployed on EAP 6

 Everything in this article applies to both EAP 6 and AS 7

On RHQ forum, we have been asked quite a few times how to monitor MBeans of a Java EE application deployed on EAP 6.

Users are often confused because on one hand, all the goodies from the JMX plugin need a JMX connection while, on the other hand, EAP 6 plugin communicates with managed servers over an HTTP management connection.

JMX connection to EAP 6

A JMX connector is accessible over the management-native port (9999 by default) on standalone servers and over the remoting port (4447 + server port offset) on managed servers (domain mode).

A custom protocol is used ("remoting-jmx") so the JMX service url will start with "service:jmx:remoting-jmx://" and EAP6_HOME/bin/client/jboss-client.jar has to be in the client code's classpath.

For authentication, standalone servers rely on the ManagementRealm, and Managed Servers on the ApplicationRealm.

If you correctly configured your server, you should be able to see your MBeans in JConsole at this point (see Using jconsole to connect to JMX on AS7).

Custom plugin requirements

So we want RHQ Agent to auto-discover our application MBeans and the corresponding resources to show up under the parent standalone server or the parent managed server.

For that, we need to create a custom plugin extending the EAP 6 plugin (see Plugin Injection Extension model). The plugin descriptor will contain a line like:


We also need component and discovery component classes from the JMX plugin, but  RHQ prevents plugin developers to use classes from more than one dependency. So there's no other choice here, the custom plugin will have to embed the JMX plugin JAR.

Eventually, we must create a component class (and its discovery counterpart) that will group our MBean resources and provide them with a JMX Connection.

AS7 JMX Util project

In RHQ's repository, the AS7 JMX Util project gives the root component classes for grouping MBeans resources: ApplicationMBeansDiscoveryComponent and ApplicationMBeansComponent. Here is how they work.

ApplicationMBeansDiscoveryComponent will be the discovery class of the custom "grouping" resource type. Plugins authors are required to provide a resource key and name and may provide a resource description and version by declaring plugin configuration properties of the following names:
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_KEY
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_NAME
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_DESCRIPTION
  • ApplicationMBeansDiscoveryComponent.PluginConfigProps.NEW_RESOURCE_VERSION
Alternatively, they can create a subclass of ApplicationMBeansDiscoveryComponent and override one/some/all of the following methods:
  • getNewResourceKey(Configuration)
  • getNewResourceName(Configuration)
  • getNewResourceDescription(Configuration)
  • getNewResourceVersion(Configuration)
By default, application MBeans will be searched with an EMS query defined in the plugin descriptor by the ApplicationMBeansDiscoveryComponent.PluginConfigProps.BEANS_QUERY_STRING plugin-config property. It is also possible to define the query string in a subclass overriding the getBeansQueryString(Configuration) method.

Plugin developers can implement their custom MBeans lookup method in a subclass overriding the hasApplicationMBeans(Configuration, EmsConnection) method.

JMX host and port discovery

The JMX server host will be detected by looking at the top level server resource plugin configuration (StandaloneASComponent or HostControllerComponent). The JMX port detection mechanism depends on the parent resource:
  • on standalone servers, the discovery component will look at the management port defined in the parent StandaloneASComponent plugin configuration and add the value STANDALONE_REMOTING_PORT_OFFSET
  • on managed servers, the discovery component will use '4447' plus the port offset of the managed server.

Authentication

JMX connectivity on EAP6 requires authentication and standalone and managed servers behaviors differ:
  • on standalone servers, the EAP 6 server will use the ManagementRealm, just as for HTTP management interface authentication. Consequently, the ApplicationMBeansDiscoveryComponent will pick up the credentials of the management user defined in the plugin configuration of the parent server resource (StandaloneASComponent)
  •  on managed servers, the EAP 6 server will use the ApplicationRealm. Consequently, there is (currently) no way for the discovery component to discover credentials automatically and it will use "rhqadmin:rhqadmin" by default.
When working with managed servers, plugin developers should subclass the discovery component and override the getCredentialsForManagedAS() method. For example, an implementation could lookup the credentials in a text file.

The custom agent plugin

The custom agent plugin must:
  • contain a plugin descriptor following the requirements described above
  • embed the JMX plugin JAR
  • embed the AS7 JMX Util project JAR
  • optionally, contain a subclass of ApplicationMBeansDiscoveryComponent 
This is very straightforward when the custom plugin is built with the Maven plugin for RHQ agent plugins: you need a POM file and a plugin descriptor (there is a sample project in RHQ repository). Let me show you what the POM file and the plugin descriptor look like:





That's it! Here are some screenshots of an example application in inventory:


"Myapp Services" grouping resource plugin configuration
"Myapp Services" grouping resource plugin configuration


Creating an "Hello Service" operation, on a managed server (domain mode)
Creating an "Hello Service" operation, on a managed server (domain mode)

Enjoy and let us now if you like it or need any help (as always, on RHQ forum or the users mailing list).

Monday, March 17, 2014

Announcing Simone project, a Java library for system monitoring

In English, pronounce sə-mōn'
Today, I pushed to Github the initial commit for Simone, a Java library for system monitoring, with no JNI code. Native invocation, when needed, is provided by JNR and friends (JNR Posix, … etc)

This work is experimental.

Currently, you can get information on CPUs, Memory and Processes on Linux systems. Next step is filesystems and networks statistics. Windows and Mac OS X are planned as target platforms.

Give it a try!

It's not yet available in any Maven repository so you’ll have to build it yourself.

Any issues? Questions? Feel free to report with GitHub issues.

EDIT: I forgot to point to the Simone project on GitHub