Wednesday, June 19, 2013

OSB Reporting using Report action (OSB Reporting/Analytics –Part2)



              The Reporting Service will be one of the framework services within SOA environment. The Reporting Service will be a generic service which will listen to Reporting queue of OSB, log or record transaction or exception data, or to raise a notification or alert to an end user or technical
Support group.
How to build a custom Reporting handler has been explained in detail in Zoggel’s blog

        The options are useful in case you have custom OSB Reporting Data Handler (Reporting Service) which will be listening to the OSB JMS Provider QUEUE. OSB will perform reporting using the OSB JMS Provider QUEUE and message payload will contain ReportData and ReportMetadata. These variables can be XML elements or Strings. These will need to be retrieved in the custom OSB reporting handler and processed further.

I will mention 2 approaches for logging/processing of report data
1.         You can report all the parameters separately as key-value pair
         
The Report Metadata will contain all the key-value pairs in side labels tag.
<ReportMetadata = <rep:messagecontext xmlns:rep="http://www.bea.com/wli/reporting"><rep:content-encoding>iso-8859-1</rep:content-encoding><rep:labels>QUERYSTRING=id=100;PROJECT_NAME=GEOTEST;;MODULE_NAME=HR;SERVICE_NAME=searchById;CORRELATION_ID=sdsadad123213123asdasd;START_TIME;SERVICE_TYPE;HTTP_METHOD</rep:labels><rep:inbound-endpoint name="ProxyService$GEOGateway$REST$v1$GEORestfulGateway"><rep:service/><rep:transport><rep:uri>/geo_services/GEOBus/GEOGateway/REST/v1/GEORestfulGateway</rep:uri><rep:mode>request-response</rep:mode><rep:qualityOfService>best-effort</rep:qualityOfService></rep:transport></rep:inbound-endpoint><rep:origin><rep:state>REQUEST</rep:state><rep:node>GatewayPipeline</rep:node><rep:pipeline>GatewayPipeline_request</rep:pipeline><rep:stage>CheckRegistryStage</rep:stage></rep:origin><rep:timestamp>2013-05-30T18:19:08.086-07:00</rep:timestamp></rep:messagecontext>>
2.       You will need to write an Xquery to populate the variable with the REPORT_DATA data structure.

Code snippet of Xquery
declare function xf:PopulateReportData($inbound as element(*))
    as element(ns0:REPORT_DATA) {
         <ns0:REPORT_DATA>
             <ns0:PROJECT_NAME>GEO</ns0:PROJECT_NAME>
            <ns0:MODULE_NAME>{substring-before(data($inbound/ctx:transport/ctx:request/http:relative-URI),"/")}</ns0:MODULE_NAME>
            <ns0:SERVICE_NAME>{substring-after(data($inbound/ctx:transport/ctx:request/http:relative-URI),"/") }</ns0:SERVICE_NAME>
            <ns0:SERVICE_TYPE>REST</ns0:SERVICE_TYPE>
            <ns0:HTTP_METHOD>{data($inbound/ctx:transport/ctx:request/http:http-method)}</ns0:HTTP_METHOD>
            <ns0:USER_NAME>{xf:GetHttpQueryParameter($inbound/ctx:transport/ctx:request/http:query-string/text(),'geo-username')}</ns0:USER_NAME>
            <ns0:USER_ID>{xf:GetHttpQueryParameter($inbound/ctx:transport/ctx:request/http:query-string/text(),'geo-userid')}</ns0:USER_ID>
            <ns0:START_TIME>{fn:current-dateTime()}</ns0:START_TIME>
            <ns0:SYSTEM_ID>{xf:GetHttpQueryParameter($inbound/ctx:transport/ctx:request/http:query-string/text(),'geo-key-id')}</ns0:SYSTEM_ID>
            <ns0:CORRELATION_ID>{fn-bea:uuid()}</ns0:CORRELATION_ID>
            <ns0:MSG_ID>{fn-bea:uuid()}</ns0:MSG_ID>
            <ns0:QUERYSTRING>{$inbound/ctx:transport/ctx:request/http:query-string/text() }</ns0:QUERYSTRING>
            <ns0:STATUS>REQUEST_RECIEVED</ns0:STATUS>
             <ns0:PROCESS_FLOW_STAGE>REQUEST</ns0:PROCESS_FLOW_STAGE>
          </ns0:REPORT_DATA>
};

In this case ReportData will contain the payload in XML format.
Report Data
<head:REPORT_DATA xmlns:head="http://geo.com/report/header"><head:PROJECT_NAME>GEO</head:PROJECT_NAME><head:MODULE_NAME>Employee</head:MODULE_NAME><head:SERVICE_NAME>RequestEmployeeId</head:SERVICE_NAME><head:SERVICE_TYPE>REST</head:SERVICE_TYPE><head:HTTP_METHOD>GET</head:HTTP_METHOD><head:USER_NAME>george</head:USER_NAME><head:USER_ID>geo123</head:USER_ID><head:START_TIME>2013-05-30T18:19:08.080-07:00</head:START_TIME><head:SYSTEM_ID>DAD</head:SYSTEM_ID><head:CORRELATION_ID>d10e6395-da61-44cf-b004-a4b0360bf965</head:CORRELATION_ID><head:MSG_ID>e52c1b84-4da9-4134-9c5f5ccd7bb8778</head:MSG_ID><head:QUERYSTRING><![CDATA[requesterFirstName=George&requesterLastName=Thomas&requesterEmail=george.Thomas@geo.com&userID=geo]]></head:QUERYSTRING><head:STATUS>SUCCESS</head:STATUS></head:REPORT_DATA>


References :

Saturday, June 15, 2013

How to use OSB Reporting Service with BAM -Part1


            The Reporting Service will be a generic service which will listen to Reporting queue of OSB, log or record transaction or exception data, or to raise a notification or alert to an end user or technical support group. It will persist report event data to a database or other similar sink so that it will be available for reporting and auditing purposes.
OSB Requesters will access the Reporting service via a non-blocking, asynchronous call, passing a simple XML log event object containing various service identification and log description data. On the service provider side, log events will be processed sequentially.

 An event can be defined as an action capable of causing more than one action to fire – so a serious exception may result in the data being persisted to database and an email notification being sent to the appropriate support team. This can be designed and developed by leveraging the functionalities of Oracle BAM tool.


Requirements ::   Reporting / Analytics

  • Ability to report service consumption per service per application
  • Calls per sec/min/hour/day
  •  Error Counts
  •   Performance Metrics
  • Performance monitoring on service level.

  Approach

                        OSB defines an extensible Reporting Framework that permits adding custom providers. This OSB/Oracle BAM integration approach consists of developing a custom reporting provider that enables report information to be sent out to Oracle BAM. This is accomplished by configuring the reporting action inside an OSB service flow, and Oracle BAM picking up the message using Enterprise Message Sources.
Oracle BAM defines a basic component called a data object. A data object represents a data entity whose information is to be stored in Oracle BAM. Each new message to be integrated into Oracle BAM must be validated against an underlying XML schema specific for the selected data object

Data Model

      Define a Reporting data structure. This will hold all the data that is defined to hold information that can help in monitoring as well as reporting.
Sample data structure
Element
Description
PROJECT_NAME
Project Name
MODULE_NAME
Module name
SERVICE_NAME
getEmployee
SERVICE_TYPE
SOAP or REST
HTTP_METHOD
GET or POST
USER_NAME
User related name
USER_ID
User related id
START_TIME
Start time of the request
SYSTEM_ID
External system Id
CORRELATION_ID
Uniquely identify the record ,so that status updates can be related to the same record
MSG_ID
Uniquely identify the record
REPOSITORY_NAME

QUERYSTRING
For rest calls to store query string
PAYLOAD
XML payloads
STATUS
Success /Error
PRIMARY_ID
Any Ids that we want to track like EmployeeID
TOTAL_TIME_SEC
Time difference between Start /End time
END_TIME
End time of the request
PROXY_ID

ERROR_TEXT
Error Messages
PROCESS_FLOW_STAGE
Request ,Response, service stage-Transcoding
TECH_COMPONENT
OSB, BPEL, Java

Sample XSD
<?xml version="1.0" encoding="windows-1252" ?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:report="http://geo.com/report/header"
            targetNamespace="http://geo.com/report/header"
            elementFormDefault="qualified">
  <xsd:element name="REPORT_DATA">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="PROJECT_NAME" type="xsd:string"/>
        <xsd:element name="MODULE_NAME" type="xsd:string"/>
        <xsd:element name="SERVICE_NAME" type="xsd:string"/>
        <xsd:element name="SERVICE_TYPE" type="xsd:string"/>
        <xsd:element name="HTTP_METHOD" type="xsd:string"/>
        <xsd:element name="USER_NAME" type="xsd:string"/>
        <xsd:element name="USER_ID" type="xsd:string"/>
        <xsd:element name="START_TIME" type="xsd:string"/>
        <xsd:element name="SYSTEM_ID" type="xsd:string"/>
        <xsd:element name="CORRELATION_ID" type="xsd:string"/>
        <xsd:element name="MSG_ID" type="xsd:string"/>
        <xsd:element name="REPOSITORY_NAME" type="xsd:string"/>
        <xsd:element name="QUERYSTRING" type="xsd:string"/>
        <xsd:element name="PAYLOAD" type="xsd:string"/>
        <xsd:element name="STATUS" type="xsd:string"/>
        <xsd:element name="QUERY_KEY" type="xsd:string"/>
        <xsd:element name="PRIMARY_ID" type="xsd:string"/>
        <xsd:element name="TOTAL_TIME_SEC" type="xsd:string"/>
        <xsd:element name="END_TIME" type="xsd:string"/>
        <xsd:element name="PROXY_ID" type="xsd:string"/>
        <xsd:element name="ERROR_TEXT" type="xsd:string"/>
       <xsd:element name="PROCESS_FLOW_STAGE" type="xsd:string"/>
        <xsd:element name="TECH_COMPONENT" type="xsd:string"/>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

Implementation
These options are useful in case you have custom OSB Reporting Data Handler which will be listening to the OSB JMS Provider QUEUE. OSB will perform reporting using the OSB JMS Provider QUEUE and message payload will contain ReportData and ReportMetadata. These variables can be XML elements or Strings. These will need to be retrieved in the custom OSB reporting handler and processed further.

BAM Dashboards


OSB has inbuilt features to monitor service’s health using metrics and alerts; however, this monitoring is mostly oriented toward technical users.    Oracle Business Activity Monitoring is the preferred product to give business users the functional monitoring on their business services and processes in the enterprise

·         BAM can be leveraged to build reports based on the data in the Reporting table.
·         The BAM dashboards are customizable and can define new dashboards as required.
·         It will be constrained by the way we define the data model, so will need to be sure we capture all the information required into the data model.

Notes
  •  These can be implemented at Gateway level to capture the request and response so all processes can be monitored.
  •  All process developers must be following naming standards in endpoint URI creation, so that we can identify the Module /service.
  •  The correlation Id needs to propagate across the proxies so that in case of errors the same record can be updated with the Error status /error message.
  •  Correlation Id can be transmitted using custom user headers in transport headers or as part of SOAP headers.
Tips on reporting in blog
References :