Mediation and Enterprise Service Bus A position paper

ESB tries to isolate the coupling between the service called and the transport medium.

ESB brings flow-related concepts such as transformation and routing to SOA. Flexibility in the transformation layer + easy connection between services.

ESB requires to provide the following characteristics:
invocation, routing, mediation, messaging, process choreography, service orchestration, complex event processing, QoS, management.

----
Paper

ESB is a mediation solution: early mediation solutions evolved to enhance the gloabl quality of services provided by large scope of DB systems. ESB uses mediation to facilitate the design of applications based on Web Services.

Wiederhold in [8, 9], mediation is "a layer of intelligent middleware services in information systems, linking data resources and application programs".

a mediator as "software module that exploits encoded knowledge about certain sets or subsets of data to create information for a higher layer of applications. It should be small and simple, so that it can be maintained by one expert or, at most, a small and coherent group of experts"

types of mediators:

• examiners -> content body for validation, authentication, etc.
  
• transformation mediators -> content body for data type transformation
  
• transcoder mediators -> modify the format and not the content. helps to go through different protocols
  
• cache mediators
  
• routers
• operator mediators -> comparators, aggregators, etc.
  
• clone mediators -> dispatch a unique request to several services

ESB is more than a mediator. It also provides

• a trading service in order to find services
• commuication service (mostly synchronous with MOM and pub/sub)
• orchestration service (based on BPEL)

Mediation in ESB includes

• Security
• Dynamic Routing and dispatch of requests (load balancing, responding to data source failure)
  
• other non-functional actions related to QoS management -> quality measurement, tracing, caching, failure detection, recovery.

Apache Synapse -> mediation for web services

Open-source ESBs

• Celtix
• Petals
• Objectweb
• Mule
• ServiceMix
• OpenESB

Java Enterprise Service Bus API (JBI) doesnt closely deal with mediation but provides standardization on exchanges between services.

with workflow languages we can model the our application but we cant serparate the proxies from the application model. as a consequence any change in the implementation will aslo affect the model and vice versa, even though in many situations it is not desirable.
Looking at the picture above, the question is whether an event processing engine handles the process of mediation and to what extent?

designing a mediation element should be kept separate from composing a mediation element.

Mediation tool development is used. This tool allows to

• describe mediation chains with ADL
• describe the execution environment
• automate the deployment and administration

This is a very interesting image as it is very close to things that we have in mind with mediation and enabling composition of services through mediators.

ESB combined with a pub/sub middleware seems to be the way to go towards this direction. In this sense mediator is a component that is able to receive 1 to n pieces of data and send 1 to n pieces of data. it can be seen as a binding between clients (1..n) and services (1..n).

A mediation chain is close to the concept of partnerlink in BPEL.

OSGi can help ESB with improving dynamism. so mediators can be defined and implemented as OSGi bundles to be dynamically loaded to the ESB infrastructure to enable communication between services of different types.

** on adopting content-based routing in service oriented architectures

WS-Eventing and WS-Notification are defined in SOA to introduce asynch notifications among web services.

WS-Notification has the following parts

• WS-Base Notification
• WS-Brokered Notification
• WS-Topic
  

WS-Notification gets close to WS-Eventing when it comes to WS-Base Notification which introduces the core roles of

• notification producer
  
• subscription manager: enables a subscriber to pause/resume/cancel a subscription
  
• subscriber
• notification consumer

WS-Brokered Notification introduces concept of notification broker rather than direct communication between the source and the sink of messages.

ESB also supports Content-Based Routing (CBR) in order to transfer messages from one service to another in a B2B type of interaction. Messages can be routed based on some pre-defined rules.

Mule and IBM Web Sphere are examples of ESB messagebrokers.

** Addressing QoS for pub/sub and CBR systems is a challenge

Different discovery approaches

• SeCSE: an apprach based on facets for discovery of services in different phases
• DIRE is the publication infrastructe developed for SeCSE
• in DIRE registries can hold any type of service
• REDS is used as a pub/sub mechanism to replicate service descriptions
• Service descriptions are shared among all registries subscribed to a topic
  
• WS-Discovery proposal
• IP multicast -> only applicable to large scale systems
• Meteor-S uses MWSDI and JXTA to provide p2p network of UDDI registries
  

overall two methods for service discovery

• replication of service descriptions
• propagation of queries within the network of registries

Vision of SOA promotes an environment which is GLOBAL and OPEN for service providers to offer their services and consumers to access and use them. Composition at runtime is also promoted to enable dynamic adaptation to changes in the environment.

--> PAPER ARGUMENT: a scalable service discovery infrastructure to allow different organizations to offer and access services globally, complemented by a pub/sub infrastructure to suit the needs of those systems that have an inherently asynch behavior and to monitor the environment.

------------

REDS is a framework of java classes to easily build a modular CBR infrastructure.

• Defining message and filter
  
• Routing Strategy can be modified
  

There are three challenges in CBR:

• the matching challenge
• the security challenge
• the reconfigurability challenge

Open issues:

• QoS for the middleware (e.g., how messages are delivered FIFO, random , etc.)
• Dependability vs Scalability
• Expresiveness vs Scalability
• Reflectiveness
• Context-awareness: context information to be used as notifications
  

A taxonomy of quality of service aware adaptive event dissemination middleware

This paper has to be read later in more details. It provides an overview of EBMs and the set of QoS requirements that they need to admit to.

Probelm that paper tries to address:

few middleware options provide support for nonfunctional service guarantees. There is a lack of comprehensive survey that analyzes all the EDMs providing support for QoS.

some of the QoS requirements for the EBM or EDM

• security
• load balancing
• reliability
• fault tolerance
• ordering
• semantics delivery

The event model has three message types

• advertisement
• notification
• subscription

There are two types to composite events

• temporal: time dependencies between primitive events.
  
• spatial: conjunctive/disjunctive combination of individual events
  

michlmayr 2008 - two papers

1. publish/subscribe in the VRESCO SOA Runtime

The overall architecture for VRESCO looks very similar to a combination of ReCoIn and OSGiBroker. Client Programs make a call to the system using DAIOS and the client library. VRESCO runtime provides support for publishing/subscribing/querying/notification using the engines that VRESCO offers.

Also, persistence is supported for storing events into the DataBase and for querying a history of events, etc.

Below is a snapshot for VRESCO event architecture, which is again very close to the Event architecture that OSGiBroker offers
VRESCO makes use of ESPER event processing engine in order to enable filtering of events and managing events when it comes to their collection and aggregation to infer some meanings from the published events.

Listeners are Esper EQL commands that request for a specific type of event to be delivered to the subscriber registered with the event processing engine.

Persistence is supported in their VRESCO pub/sub system by using NHibernate engine which persists all the events. The events history can then be queried for specific set of exchanged events.


2. Advanced Event Processing and Notifications in Service Runtime Environments

VRESCO supports the following set of events

• Binding and Invocation Events
• Query information events
• user information events

They consider two types of external consumers

1. human
2. services
3. WS-Eventing
4. WS-Notification

Even Ranking

• Priority-based
• Hierarchical
• root events have higher importance compared to leaf events
• Type-based
• Content-based (the keyword exception is more important than warning)
  
• Probability-based (frequent events are less important than infrequent events)
  
• Event Patterns
  

Event-correlations are used to avoid losing track of events and their relationships. something like an event identifier can be used to correlate events to one another.
I think event correlation can be done either by scanning the content for events or by looking into channels and separating events based on the channels they get published to.

Correlation sets enable users to track all relevant events for a correlation ID without losing the track of what is happening between these correlation sets.

VRESCO Web service Eventing specifies 5 different operations

• subscribe
• unsubscribe
• renew
• expires
• subscription ends

important: due to the large set of published events, relational databases are not always preferrable as a more efficient indexing strategy is required. Vector space engine as described in [17] might be a better choice. The advantage is that the search returns a list of fuzzy matches together with a similarity rating.

Throughput for different methods of publishing events can be measured. Also, since we are using caching, it is easier to see how throughput affects the overall behavior of the system. As for events, we can also store all the events in the DB and query the DB only in situations needed. This decreases the chance of having low preformance as a result of dealing with the relational DB.

Their related work is very interesting and important. It is a MUST READ set of papers.

Henssen 2008 - QoS attributes

Capturing and Using QoS Relationships to Improve Service Selection

Promotes the use of MCDA (multi-criteria decision Analysis

• important to analyze weights related to criteria + weights related to interactions between criteria
  
• 
  

Dealing with Quality Tradeoffs during Service Selection

aggregation approaches for integrating weights into the service selection procedure for a QoS:

1. compensatory [29][30]: they amount to being substitution rates. The priorities for different criteria to be expressed on the same scale
   
2. noncompensatory [4, 8, 21, 31]: weights are simply a measure of relative importance of the criteria involved. They are only used to indicate the relative importance
   

Promotes the use of outranking methods for defining global priority constraints

outranking relation is a binary relation S on the set of potential choices A such that aiSaj to decide that ai is at least as good as aj.

Pj(a, b) = Fj[dj(a, b)] for all a, b
dj(a, b) = gj(a) - gj(b)

where gj(a) is the score of service a over quality of service a

There are six categories of functions for F

1. immediate preference
2. indifference threshold
3. increases continuously until reaching the indifference threshold
4. comprises an indifference and a preference threshold
5. increases continuously between an indifference and a preference threshold
6. Gaussian law with a fixed standard deviation

• aggregating the preferences
• wj is the preference for characteristic j
  

• outranking flows
• The positive outranking flow expresses how an alternative a is outrankking all the others (n-1 alternatives)
  
• The negative outranking flow indicates how an alternative a is outranked by other n-1 alternatives

complete ranking of PROMETHEE method is derived using the following formula

The paper is clumsy when it comes to their real experiment. values are drawn from no where and it is not clear how these values are calculated. The author needs prior knowledge and there is a lack of proper description

Monitoring the QoS for Web services

QoS metrics

• Provider-advertised (execution price)
  
• Consumer-rated (service reputation)
  
• Observable
• IT Level
• Business Level

The QoS value needs to be recomputed whenever the execution of a service instance is computed

observational model

• service monitoring architecture
• QoS metric computation
• high volume of service operational events
• complexity of metric computation
• metric value persistence

computing/updating metric values in realtime using a high performance metric computation engine

contributions of the paper:

• monitoring-enabled SOA infrastructure
• declarative event detection
• event routing
  
• mointoring QoS with small programming efforts
• Efficient QoS computation
• compilation interpretation approach
• improve event processing throughput
• custom executable ECA rule at buil time
• observatoin model is transformed to invoke generated code
• model driven planning to enable wait free concurrent threading

QoS is a broad concept encompassing a large number of context-dependent and domain-specific nonfunctional properties.

1. Process Monitor Context
2. Service Mointor Context (Service Interface Monitor Context)
   
3. QoS metrics

Event (eventPattern)[condition]|expression

• eventPattern = service operational event / change in the metric
• condition = circumstance to fire an event
• expression = association predicate + value assignment
  

Metric computation engine takes observation models as input and generates event subscriptions for the semantic pub/sub engine. Thus, the events from one service engine are delivered to another service engine.

High Performance Metric Computation for QoS Metrics

ECA rules are mapped to a state chart with the transitions from events to metrics or metrics to metrics.

There are two parts to execution of state charts.

• Interpretation of state chart logic
• Interpretation of the expressions within the states
• Thread scheduling for executing events
  

Examples of Mashups

IBM's QED Wiki
Yahoo Pipes
Google Mashup Editor
Microsoft's Popfly

Semantics identified by Web services community

• data (I/O)
• functional (behavioral)
• nonfunctional (QoS, policy)
• execution (runtime, infrastructure, exceptions)

Semantic Annocation of Web Services

• hREST
• SA-REST

Mashups available on the Web

• ProgrammableWeb
• APIHut

A Model for Web services Discovery with QoS

Ran, S. 2003. A model for web services discovery with QoS. SIGecom Exch. 4, 1 (Mar. 2003), 1-10. DOI= http://doi.acm.org.proxy.lib.sfu.ca/10.1145/844357.844360


Service Supplier -> provides Certifier with QoS information -> Certifier either accepts or downgrades the information -> returns the result back to the supplier -> supplier registers the service (the functional descriptions + the certified QoS) + UDDI checks with certifier the QoS.

Web services are provided by third parties and are invoked dynamicall over the intenet and thus their QoS can vary greatly.

A framework is need to capture QoS provided by supplier and QoS required by customer and try to provide a match between the two.

ISO 8402 Description for Quality: the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.

QoS: a set of non-functional attributes that may impact the quality of the service offered by a
Web service

Categories for QoS

1. Runtime related quality of service

• Scalability (related to throughput and performance)
  
• Capacity (concurrent requests)
• Performance ( speed in completing a service request)
• Response time
• Latency
• Throughput (number of completed service requests over a time period)
• Reliability
• Mean Time between Failuers
• Mean Time to Failure
• Mean Time To Transion
• Availability
• Robustness / Flexibility
• Exception Handling
• Accuracy

2. Transaction Support

• Integrity ACID ->
  
• Atomicity -> entirely executes or not at all
• Consistency -> maintains integrity
  
• Isolation -> runs as if no other transactions are present
  
• Durability -> the results are persistent

3. Configuration Management and Cost Related

• Regularity ( how well the service is aligned with the regulations)
  
• Supported Standard (if the service complies with standards)
  
• Stability / Change Cycle (frequency of change in the service in terms of interface)
  
• Cost ( the cost for using the service)
  
• Completeness (the difference between the specified set of features and the implemented set of features)

4. Security related QoS

• Authentication
• Authorization
• Confidentiality
• Accountability
• Traceability + Auditability
• Data Encryption
• Non-repudiation (A Principal can not deny requesting the service after the fact)