Business Activity Monitoring



Business Activity Monitoring
Published on April 2014
Author: Rupesh Chintapalli ProjectLead IMS
Reviewed By:


Applied To BizTalk Server 2013 BAM
Contents:
BizTalk BAM – Concepts----------------------------------------------------------------------------------1
BAM Management Utility Commands------------------------------------------------------------------6
BAM – SQL Server Views in the BAM Primary Import Database-------------------------------9
BAM – The BAM Databases----------------------------------------------------------------------------10

Using OrchestrationEventStream to write BAM from BizTalk orchestration----------------11

Using MessagingEventStream to write BAM Events from pipeline components---------13

Limitation of Tracking Profile Editor (TPE) ---------------------------------------------------------14

BAM Tools---------------------------------------------------------------------------------------------------15
Tracking data in the BAM portal ----------------------------------------------------------------------17
Enabling the BAM Excel Add-In for Excel 2012---------------------------------------------------19
Scenario Study---------------------------------------------------------------------------------------------21

 



BizTalk BAM – Concepts
Information workers need flexibility in looking at and evaluating business processes. A purchasing manager might need to see how many POs are approved and denied each day, for example, while a sales manager might want an hourly update on what products are being ordered. Meeting these diverse needs requires a general framework for tracking what’s going on with a particular business process. This is exactly what the Business Activity Monitoring (BAM) component in Microsoft BizTalk Server provides.
As the figure above illustrates, the BAM component allows monitoring of events and data produced by a BizTalk application. This information is made accessible using SOAP-callable Web services, and it can be accessed in several different ways, as follows:
  • Through Microsoft Excel or other desktop clients, such as a custom dashboard application.
  • Using a BAM portal, a component in BizTalk Server that enables business users to examine and configure BAM information. Using the BAM portal, an information worker can select a particular instance of some business process, and then choose a specific BAM view into the process. Each of these views provides a different perspective, such as graphical depictions of per-product sales trends or current inventory levels, or other key performance indicators. The information in these views might be updated every day, every hour, or more frequently. Using the BAM portal, an information worker can also define aggregations of data, such as the number of orders filled, canceled, or in progress over the last hour. Implemented as a set of ASP.NET pages, the BAM portal can also be hosted as a Web part inside Windows SharePoint Services.
  • Through SQL Server Notification Services, allowing BAM information to be delivered as notifications. While the first two options allow information workers to examine BAM information, this third option creates a notification when something interesting happens. Using the BAM portal’s alert manager, an information worker can define alerts that will be sent when a specified event occurs. For example, a BAM user might choose to send an e-mail to a particular manager whenever the number of canceled orders in a day exceeds ten, or perhaps inform certain sales associates any time an order arrives from their largest customer.
Under the covers, each BAM view relies on one or more BAM activities. A BAM activity represents a specific business process, such as handling purchase orders or shipping a product, and each one has a defined set of milestones and business data. For example, a purchase order activity might have milestones such as Approved, Denied, and Delivered along with business data like Customer Name and Product.
For information workers accessing BAM through Excel, BAM activities and BAM views can be created using an Excel add-in. The BAM Activity Wizard for this add-in allows the defining activities, while the BAM View Wizard allows the defining of views based on those activities. In fact, the BAM View Wizard really just helps an information worker build a standard Excel PivotTable using the information in one or more BAM activities. The information that this view provides can then be shown directly in Excel, as the figure below illustrates.
In this simple example, two Excel charts display information about order progress and sales. A BAM view can be more complex, and its creator can specify which users are allowed to see the data it exposes. Maybe a purchasing manager can access certain things in a view into the purchase order process, for instance, that are hidden from purchasing clerks.
While information workers can create BAM views and BAM activities on their own, these views and activities depend on information provided by the orchestrations they monitor. Accordingly, developers still have a role to play. Using a tool called the Tracking Profile Editor (TPE), a developer must configure an orchestration so that it provides the information required for a particular BAM activity, and thus for the BAM views that depend on this activity. This tool allows a developer to graphically associate the appropriate events and message fields in an orchestration with corresponding milestones and business data in a BAM activity. The BizTalk Server engine then sends these events and message field values to the Tracking database, as shown in the earlier figure, where they can be accessed by the BAM component. While developers must play their part, BAM activities and BAM views aren’t their concern. These business-oriented services are created, maintained, and used solely by information workers.
In BizTalk Server, the TPE can also be used to specify how pipelines generate events. More important, BAM can now accept and display events generated by any user code, whether or not it is built as an orchestration. Any application built using the .NET Framework can potentially be monitored using the BAM component of BizTalk Server.
What is BAM used for?
BAM is used to monitor business milestones and key metrics in near real-time throughout a process in BizTalk.
Description
Business Activity Monitoring (BAM) provides visibility on business processes by tracking process milestones and business data (KPIs). BAM allows business users and decision makers to gain insight on the current health of in-flight processes they are responsible for. Up-to-date information is accessible to users via Microsoft Office BI tools or a designated portal and through automatic business alerts. This allows users to streamline and improve daily business operations by proactively escalating issues and mitigating problems sooner.
BAM is composed of a collection of tools and runtime components to layer business tracking without having to write any code. A set of runtime interceptors capture interesting business events inside of BizTalk Application or other applications and publish those events into the BAM infrastructure.
It also provides a set of tools for managing this infrastructure, tools for aggregating and viewing the data as well as APIs to optionally publish data to through custom code.
BAM is tied directly to the central BizTalk engine (that is, processes and databases) through OLAP cubes. These cubes are created automatically when a developer deploys a BAM report and profile, typically by using a combination of Microsoft Excel pivot tables and the BizTalk Tracking Profile Editor.
BAM Activities
BAM activities identify the milestones and tracking data an individual is interested in tracking. Milestones are the steps in an activity that are measured in time, and tracking data is the key data points in a process you are interested in tracking (such as a customer ID or name).
BAM View
A BAM view is a representation of the milestones and business data tracked in one or more activities.
  • The Group milestone allows related milestones to be treated as a single milestone. For example, grouping the EndSuccess and Exception milestones is useful to indicate the completion of the process regardless of whether it completed successfully or failed.
  • A duration calculates the time between two business milestones and is useful when reporting the time elapsed between two milestones is important.
  • A progress dimension defines milestones and stages for a process. A progress dimension allows you to display how many processes are at an existing status at a given time. For example, a progress dimension will display how many processes are in the middle of execution, how many encountered an exception, or how many processed without an exception.
  • Alias Allows the referencing of a previously created milestone
BAM alerts
BAM alerts allow you to configure and receive alerts related to specific changes in business data. Alerts are set up per BAM view.
BAM is based on SQL Server Reporting Services, and events are fundamentally triggered via SQL Server Notification Services.
Deployment
Use the deploy-all command to deploy the BAM activity and view defined in your BAM Excel workbook:
1
C:\Program Files\Microsoft BizTalk Server 2006\Tracking\BM.exe deploy-all-DefinitionFile:"MyBAMWorkbook.xls"
This command reads the BAM Excel workbook specified in the DefinitionFile parameter and creates SQL Server database artifacts based on the activities and view defined in the spreadsheet.
BAM Management Utility Commands
You can use the BAM utility to perform the following tasks:
  • Consume BAM definition and BAM configuration XML as input. The BAM definition XML or XLS files define the data to track and aggregate, as well as the business end user’s view on the tracking data. The BAM configuration XML mandates where to deploy the infrastructure, such as the server name, database name, and other database settings.
  • Deploy the run-time infrastructure on the server, which includes the BAM Primary Import database, BAM Star Schema database, BAM Analysis database, and corresponding Data Transformation Services (DTS) packages. The following items are created during this step:
      • BAM Primary Import database
      • BAM Star Schema database
      • BAM Archive database
      • BAM Analysis database
BAM utility (bm.exe) does many things for you during the deployment of a XML or XLS file, like creating SQL tables, indexes, triggers, views, stored procedures, OLAP cubes, DTS packages, SQL and OLAP security roles etc.
How to use
In a very simply way:
To see the list of all commands or in other words, the help of the command, you can put:
1
bm.exe
or
1
bm.exe help

Note: You have to browse to the following directory: <installation path>\Program Files\Microsoft BizTalk Server %version%\Tracking.( where’s %version% can be 2016, 2009, 2010…)
– COMMANDS –
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
help
setup-databases, migrate-sql
get-config, update-config
get-references, enable-reference, disable-reference
deploy-all, update-all, remove-all
get-activities, remove-activity
get-views, remove-view
get-alerts, remove-alerts
get-defxml
get-accounts, add-account, remove-account
get-subscriptions, add-subscription, remove-subscription
enable-alerts, disable-alerts
get-index, create-index, delete-index
get-activitywindow, set-activitywindow, get-rtawindow, set-rtawindow
get-changes
update-livedataworkbook, regenerate-livedataworkbook
To get detailed help for a specific command:
“bm.exe help -Command:<command>” (that there is no space after “-Command:”)
For example:
1
bm.exe help -Command:get-config
List of Commands
Here is a table with the bm.exe commands in BizTalk Server 2013:
BM utility command
(BTS 2006)
Example
bm.exe setup-databases
bm.exe setup-databases -ConfigFile:BamConfiguration.xml
bm.exe migrate-sql
bm.exe migrate-sql -From:sql2000 -To:sql2005 -NSUser:domainuser1 bm.exe migrate-sql -From:sql2000 -To:sql2005 -Server:MyServer -Database:db1
bm.exe get-config
bm.exe get-config -FileName:MyConfig.xml
bm.exe update-config
bm.exe update-config -FileName:MyConfig.xml
bm.exe get-references
bm.exe get-references -Server:MyServer -Database:MyPrimaryImport
bm.exe enable-reference
bm.exe enable-reference -TargetServer:MySrv -TargetDatabase:BamPrimaryImport
bm.exe disable-reference
bm.exe disable-reference -TargetServer:MySrv -TargetDatabase:BamPI
bm.exe deploy-all
bm.exe deploy-all -DefinitionFile:SalesManagerView.xls
bm.exe update-all
bm.exe update-all -DefinitionFile:SalesManagerView.xls
bm.exe remove-all
bm.exe remove-all -DefinitionFile:SalesManagerView.xls
bm.exe get-activity
bm.exe get-activities -View:SalesManagerView
bm.exe remove-activity
bm.exe remove-activity -Name:PurchaseOrder
bm.exe get-views
Bm.exe get-views
bm.exe remove-view
bm.exe remove-view -Name:SalesManagerView
bm.exe get-alerts
bm.exe get-alerts
bm.exe remove-alerts
bm.exe remove-alerts -View:SalesManagerView
bm.exe get-defxml
bm.exe get-defxml -FileName:BAMDefinition.xml
bm.exe get-accounts
bm.exe get-accounts -View:SalesManagerView
bm.exe add-account
bm.exe add-account -AccountName:domainuser -View:SalesManagerView
bm.exe remove-account
bm.exe remove-account -AccountName:domainuser -View:SalesManagerView
bm.exe get-subscriptions
bm.exe get-subscriptions -View:SalesManagerView -Alert:SalesTooLow
bm.exe add-subscription
bm.exe add-subscription -View:v1 -Alert:a2 -AccountName:domainuser -Type:File
bm.exe remove-subscription
bm.exe remove-subscription -View:v1 -Alert:a2 -AccountName:domainuser
bm.exe enable-alerts
bm.exe enable-alerts -View:SalesManagerView
bm.exe disable-alerts
bm.exe disable-alerts -View:SalesManagerView
bm.exe get-index
bm.exe get-index -Activity:PurchaseOrder
bm.exe create-index
bm.exe create-index -IndexName:Idx1 -Activity:PO -Checkpoint:State,City
bm.exe delete-index
bm.exe delete-index -IndexName:Idx1 -Activity:PO
bm.exe get-activitywindow
bm.exe get-activitywindow -Activity:PurchaseOrder
bm.exe set-activitywindow
bm.exe set-activitywindow -Activity:PurchaseOrder -TimeLength:6 -TimeUnit:Day
bm.exe get-rtawindow
bm.exe get-rtawindow -View:ManagerView -Activity:PO -Rta:Rta1
bm.exe set-rtawindow
bm.exe set-rtawindow -View:V -Activity:A -Name:R -TimeLength:5 -TimeUnit:Minute
bm.exe get-changes
bm.exe get-changes
bm.exe update-livedataworkbook
bm.exe update-livedataworkbook -Name:SalesManager_Live.xls
bm.exe regenerate-livedataworkbook
bm.exe regenerate-livedataworkbook -WorkbookName:SalesManager_Live.xls
BAM – SQL Server Views in the BAM Primary Import Database
When you issue the deploy-all command using “bm.exe” tool, BAM automatically creates the underlying data model for you activities and views.
Microsoft has committed to maintaining the structure of the views in futures versions of the product, so you can read the view data using any programming tool that works with SQL Server.
The term view in the content of BAM is used with two different meanings. A SQL Server view is a virtual table that is defined using a SELECT statement. A BAM view is derived from an activity by subsetting, aggregation, or aliasing. SQL Server view are used to implement both BAM activities and BAM views
The views that are directly derived from the deployment of a BAM activity:
  • bam_<Activity Name>_ActiveInstance: Returns the list of the active activity instances based on the records stored in the active activity table.
  • bam_<Activity Name>_AllInstances: Joins the active and complete activity views, returning a list of all the BAM activity instances, in other words, return all instances of the activity regardless of state.
  • bam_<Activity Name>_AllRelationships: Joins the active and complete activity relationship tables, returning a list of all the relationships of the BAM activity instances.
  • bam_<Activity Name>_CompletedInstances: Returns the list of the completed activity instances based on the records stored in the completed activity table.
  • bam_<Activity Name>_InstancesForArchive: return all instance of the activity that will be archived on the next execution of the DTS Package.
Additionally, a new set of views are created for each activity view that we include in the BAM activity model:
  • bam_<ActivityName>_<ActivityView Name>_ActiveView:Returns the list of active activity instances including the calculations of the required dimension levels.
  • bam_<ActivityName>_<ActivityView Name>_ActiveAliasView: Return the list of items currently in the active state by alias.
  • bam_<ActivityName>_<ActivityView Name>_CompletedView: Returns the list of completed activity instances including the calculations of the required dimension levels.
  • bam_<ActivityName>_<ActivityView Name>_CompletedAliasView: Items currently in completed state by alias.
  • bam_<ActivityName>_<ActivityView Name>_View: Joins the two previous views (ActiveView and CompletedView), returning the list of all activity instances included the calculation of the required dimension levels, in other words, all items in the view.
These views enable two key functionalities in the BAM infrastructure: At the first level, activity views are the fundamental mechanism to expose BAM data to other applications, including other components of the BAM infrastructure like the aggregation process. Additionally, these views include some of the calculations required by elements such as measures that are not directly included in the activity tables.
BAM – The BAM Databases
BAMAlertsApplication and BAMAlertsNSMain
Are used by SQL Notification Services to support BAM Alerts.
  • BAM Notification Services Application database (BAMAlertsApplication) Contains alert information for BAM notifications.
  • BAM Notification Services Instance database (BAMAlertsNSMain) Contains instance information specifying how the notification services connect to the system that BAM is monitoring.
BAMPrimaryImport
The BAM Primary Import database (BAMPrimaryImport) table is primarily designed to store data for current and recently completed. You will see several tables inside:
  • the table bam_<activity>_Active and bam_<activity>_Completed contains the activities;
  • the other tables are used by relationships and continuations
BAMStarSchema
The BAM Star Schema database (BAMStarSchema) Contains the staging table, and the measure and dimension tables.
BAMAnalysis
BAM Analysis database (BAMAnalysis) contains BAM OLAP cubes for both online and offline analysis.
BAMArchive
BAM Archive database (BAM Archive) archives old business activity data. You can create a BAM Archive database to minimize the accumulation of business activity data in the BAM Primary Import database.

Using OrchestrationEventStream to write BAM Events from BizTalk orchestration

The OrchestrationEventStream (OES) is one of the artifacts developers should consider in order to programmatically populate a BAM activity model from within a BizTalk orchestration.
The OES writes BAM Activity data into the state of the BizTalk orchestration and does not require a database roundtrip each time. When the orchestration completes or reaches a persistence point, the BAM activity data is written into the message Box along with the orchestration state.
Because this data is part of the orchestration state, all data is rolled back along with the orchestration if an error condition is reached. This method gives you complete transactional integrity, unlike the other event stream options, and offers the best performance for an orchestration-based solution.
The OES API stores tracking data first in the BizTalk Message Box database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS).
It is found in the Microsoft.BizTalk.Bam.EventObservation namespace. To use the API you must add the Microsoft.Biztalk.BAM.Xlangs.dll to your project.
So in your BizTalk project, include references to the following assemblies:
Microsoft.BizTalk.Bam.EventObservation and Microsoft.BizTalk.Bam.XLANGs.
These assemblies may be found in your “Microsoft BizTalk Server 2013Tracking” folder.
The OrchestrationEventStream is utilized by the TPE tool. In BizTalk Server 2004, it was an internal component used by TPE and not exposed for general use. The transactional integrity requirement came up during a customer engagement, so it was later exposed in the 2004 release by a hotfix and is in the box with BizTalk Server 2006.
Sample code:
Note that because the OrchestrationEventStream is implemented as a static class, there is no need to manually create an event stream each time, and a connection string isn’t required, because the OrchestrationEventStream piggybacks the persistence of the orchestration.
1
2
3
4
5
6
7
8
9
10
11
//Create a new, unique activity identifier to use as the ActivityID in BAM
stringactivityId = Guid.NewGuid().ToString() + "_"+ DateTime.Now;
  
//Start the activity record identified by activityId
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity(“MyTrackingDemo”, activityID);
  
// Updates the activity record.
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity(“MyTrackingDemo”, activityID, “MESSAGE_ID”, msg(BTS.MessageID), "TRANSACTION_CREATED", DateTime.UtcNow);
  
// End the activity record.
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity(“MyTrackingDemo”, activityID);
Using MessagingEventStream to write BAM Events from pipeline components
MessagingEventStream (MES) is used inside a BizTalk pipeline component to write Bam as part of the messaging transactions ensuring that your BAM event persistence remains in sync with the BizTalk pipeline transactions.
Messaging Event Streams are asynchronous and store tracking data first in the BizTalk message Box database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS). This class is derived from the base class Event Stream.
MES is available since BizTalk 2004 with SP1.
Normally, we write the code to produce BAM events in the Execute method of the pipeline component. Execute method have two parameters:
  • An IPipelineContext: contains the pipeline context
  • An IBaseMessage: contains the message being processed.
The pipeline context has a GetEventStream method that returns a MessagingEventStream.
Code sample:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
publicIBaseMessage Execute(IPipelineContext context, IBaseMessage message)
{
   //Get the Messaging Event Stream
   MessagingEventStreameventStream = context.GetEventStream();
   …
  
   //Write to BAM
   //Create a new, unique activity identifier to use as the ActivityID in BAM
   stringactivityId = Guid.NewGuid().ToString() + "_"+ DateTime.Now;
  
   //Start the activity record identified by activityId
   eventStream.BeginActivity("MyTrackingDemo", activityId);
  
   // Updates the activity record.
   eventStream.UpdateActivity("MyTrackingDemo", activityId, "MESSAGE_ID", message.MessageID, "TRANSACTION_CREATED", DateTime.UtcNow);
  
   eventStream.EndActivity("MyTrackingDemo", activityId);
  
   // Optional Flush
   eventStream.Flush();
}
 Limitation of Tracking Profile Editor (TPE)
BizTalk artifacts
The most important limitation of TPE is that it can only be used to capture data from BizTalk artifacts. If you want to use Bam to capture data from custom code, you will need to use the BAM API.
Messages and elements limitations
Some messages and elements can’t be tracked with TPE. You can only track messages that have a schema that is known at design time, so you can’t track messages of type string or System.XML.XMLDocument.
Also you can’t capture data from repeating fields.
Some limitations exist on the length of data item names. The combined length of a folder name and data item instance value cannot be longer that 128 characters.
Orchestration shapes limitations
You also can’t track any of the following orchestration shapes:
  • Group (Task)
  • Loop (While)
  • Message Assignment
  • Suspend
  • Terminate
  • Throw Exception
  • Transform
You can work around these limitations for some shapes by wrapping them in a nontransactional scope shape and tracking the scope. However this don’t work for the Suspend and Terminate shapes, as they never fire the Shape End event.
BAM Tools:
BAM Portal
The BAM portal provides real-time, end-to-end visibility into a business process. It is a Web-based feature that consists of a collection of ASP.NET pages. You can customize BAM to enhance the performance and experience for your users. It is installed as part of the BizTalk Server installation process and configured under the BAM Portal tab.
It can be installed on the same server as BizTalk or on a separate server that runs IIS dedicated to BAM and BAM tools (recommended), as long as that server is joined to the same BizTalk Server group.
BM.EXE
BAM Management utility or “bm.exe” provides for end-to-end management and deploy of BAM
Related article:
BAM Add-in for Excel (Microsoft Office Excel)
Provides for the creation of observation models to be deployed to the BAM infrastructure, and the capabilities to consume BAM data within Excel via a data connection (known as “Live Workbooks”)
Tracking Profile Editor (TPE)
Is a graphical user interface to help developers create new tracking profiles or modify existing tracking profiles that map a specific view of internal business processes, as well as associated data, to an orchestration.
It is installed as part of the BizTalk Server installation process and configured under the BAM Portal tab.
Microsoft Office Sharepoint Server
WSS may be integrated with BAM and the BAM data stores by serving as a portal mechanism to display BAM data or by using MOSS dashboard and portal to display KPIs from the BAM API (views).
Microsoft SQL Server Reporting Services
Reporting Services may be integrated with BAM and the BAM Primary Import database to build reports against the data stored with BAM.
Other tools:
  • Orchestration Designer for Business Analysts: may also be used to define BAM activities and business process.
  • Visual Studio: Serves the means to create interceptor configuration (IC) files.
  • MS PerformancePoint Server: used to host dashboards and scorecards.
The BAM API defines four main classes:
  • DirectEventStream: used in a :NET application to do a buffered write of data to BAM.
  • BufferedEventStream: used in a :NET application to do a unbuffered write of data to BAM.
  • OrchestrationEventStream: used when writing to BAM programmatically within an orchestration; Provides transactional consistency with the orchestration.
  • MessagingEventStream: used when writing to BAM programmatically within a pipeline; Provides transactional consistency with the messaging engine.


All of these classes are derived from the base class EventStream.
Differences between BufferedEventStream and DirectEventStream
  • BufferedEventStream:
    • Is asynchronous
    • When you update a BufferedEventStream, the update is cached and written later.
    • The BufferedEventStream take advantage of the cache to improve performance
  • DirectEventStream
    • Is synchronous
    • When you execute an update, the call won’t return until the database write is commited
    • If your application can’t ever afford to lose data in server crash, use this class
Advantages of using OrchestrationEventStream
  • Performance: it can piggyback its databases writes on the orchestration persistence points for maximum performance
  • Consistency: because it writes during orchestration persistence points, even in the event of the server crash, the state of the orchestration recorded by BAM is guaranteed to be consistent with the last orchestration persistence point
The most commonly data sources used with BAM are:
  • BizTalk Application: BAM can capture data from fields in BizTalk messages, can capture milestones from the date and time a BizTalk message is received or sent and also can capture fields in the BizTalk context properties.
    Data capture from BizTalk, normally, is configured using the Tracking Profile Editor (TPE)
  • BAM API: .NET code can write data directly into BAM using an API. Code that is executed in BizTalk, normally use either the OrchestrationEventStream (in orchestrations) or MessagingEventStream (in pipelines) classes.
    Code that isn’t executing in BizTalk will use either the DirectEventStream or BufferedEventStream classes.
  • WCF Services (WCF Interceptor): BAM can capture data from SOAP envelope in a WCF request, response or fault message. The data is specified using interceptor configuration (IC) file (XML file created by developer).
    The service’s app.config must be modified to load the BAM components at runtime.
  • WF Workflows (WF Interceptor): like WCF Services, BAM can also capture milestones from WF Activities. Am IC file is used to specify the data that must be captured. BAM components must also be loaded at runtime.

BAM –Why are the times for my tracking data in the BAM portal incorrect?

BAM captures milestones for objects derived from Microsoft.BizTalk.Bam.EventObservation.EventStream in Coordinated Universal Time (UTC) format. When you send date/times to BAM using the APIs, they are received in the format sent with no conversion to UTC format.
If you use local time, the times will not be converted to UTC format and will be out of sequence relative to UTC times that are recorded.
To solve this problem, modify your data to make it conform to UTC format.
BAM API Sample:
Global.es.BeginActivity("BAMApiPo",poid.ToString());
 
Global.es.UpdateActivity("BAMApiPo",poid.ToString(),
   "Received",DateTime.UtcNow,
   "Product",xePO.SelectSingleNode("Product").InnerText,
   "Amount",xePO.SelectSingleNode("Price").InnerText);
 
Global.es.UpdateActivity("BAMApiPo",poid.ToString(),
   "Packaged",DateTime.UtcNow);
 
Global.es.EndActivity("BAMApiPo",poid.ToString());
 
 
 
 
 
 
 
BAM Excel Add-In for Excel 2012 – This workbook has lost its VBA project, ActiveX controls and any other programmability-related features
I have installed and enabled BAM Excel Add-in in Excel 2012. Now when I start Excel I get the following error message:
“Repairs to BAM.xla’
Excel was able to open the file by repairing or removing the unreadable content.
This workbook has lost its VBA project, ActiveX controls and any other programmability-related features”
CAUSE
  • You get this error because during the installation of MS Office 2007, “Visual Basic for Application” was not installed.
SOLUTION
  • To solve this you need to change your installation of MS Office.
    • Close Excel if it is open.
    • In the Control Panel choose “Add or Remove Programs”
    • Select Microsoft Office in the list and click on the Change button.
    • Choose “add or remove features” and press “Continue”
·          
    • Select “Visual Basic for Applications” in the “Office Shared Features” list and set it to “Run from My Computer”. Then click on the continue button.
After the installation is finished you will be able to use BAM Excel Add-In.
Enabling the BAM Excel Add-In for Excel 2012
Earlier versions of Office 2012 are quite different with regard to the layout of menu and some options may be hard to find in the beginning. Add-In is one of these options.
For enabling the BizTalk Server 2013 BAM Add-In for Excel 2012 (assuming that BAM and Excel are already installed), we need to add it to the ribbon so we can access it. We do that by:
  • Going to “More Commands…” option:
  • This will open the Excel options dialog box and from there you can select Add-Ins. You should select “Business Activity Monitoring” in the list:
  • Do not exit from “Excel Option” box…
    After select “Business Activity Monitoring”, select Manage: Excel Add-Ins, and press “Go…”, this will bring up a familiar dialog where you need to check the “Business Activity Monitoring“ Add-In.
  • Press “OK”
Having done this the BAM Add-In will now be accessible through the ribbon and you can edit your BAM Activities and Views.
Microsoft BizTalk Server 2013 includes several Business Activity Monitoring (BAM) samples in its software development kit (SDK). This section provides detailed information about the functionality demonstrated by each BAM sample, instructions for building and running the sample, and the results you can expect.
1) BAMAPI :
A business analyst defines interesting data using a wizard, examines aggregations, and queries activity data. An information technology (IT) administrator creates SQL, Data Transformation Services (DTS), and Online Analytical Processing (OLAP) infrastructure. Developer instruments the business implementation.
2)BAM End-to-End : Demonstrates how to correlate events from multiple components using BAM.
3BAM API from an Orchestration Expression: Demonstrates how to
Ø  Use the BAM API from a BizTalk Server orchestration.
Ø  Track repetitive items inside a message as separate activity instances.
Ø  Establish a relationship between BAM data tracked from an orchestration using a tracking profile and data tracked from the API.

BAM API

The BAM API illustrates how to incorporate calls to the BAM API into an application to save key information that you can monitor.
Scenario:
It generates purchase orders, processes each purchase order, creates shipments, and creates and processes invoices. As the sample runs, it creates and updates BAM activities to reflect the details and disposition of the purchase orders and invoices.

How This Sample Was Designed and Why

This scenario was designed to illustrate how to use BAM to store information from an application that is not a BizTalk orchestration. While the application is simple, it illustrates several aspects of BAM that you are likely to use in a production application.

These include the following:
  • Multiple threads that contribute to a single activity
  • Creating a relationship between two activities
  • Using a continuation to allow the same activity to be accessed with different IDs
The BAM API sample consists of three main classes:
1) To process purchase orders
2) To process shipments
3) To process invoices
Explanation: Each class has a Run Once method that retrieves a message from a queue, and then processes the message. Each class also has a Run method that continually calls the Run Once method.
The Run Once method of the PoApplication class does the following:
  1. Creates an XML message that represents a purchase order.
  2. Begins a BAMApiPo activity and adds to the activity information about the purchase order and when it was received.
  3. Arbitrarily approves or rejects the purchase order.
  4. Updates the BAMApiPo activity to record the status of the purchase order (accepted or rejected).
  5. If the purchase order was accepted, the Run Once method also does the following:
    1. Creates an XML message to represent a package to be shipped, and adds the message to the queue of shipments.
    2. Adds the XML message that represents the purchase order to the queue of purchase orders to be included in an invoice.
    3. Enables continuation for the BAMApiPo activity.
    4. Ends the BAMApiPo activity.
The Run Once method of the Shipment Application class does the following:
  1. Retrieves from its queue an XML message that represents a package to be shipped.
  2. Updates the BAMApiPo activity to record the time that the package was shipped.
  3. Ends the BAMApiPo activity.
The Run Once method of the Invoice Application class does the following:
  1. Retrieves from its queue an XML message that represents a purchase order to be invoiced.
  2. Begins a BAMApiInvoice activity.
  3. Creates a BAM relationship between the BAMApiInvoice activity and the BAMApiPo activity for the purchase order that is being invoiced.
  4. Adds to the BAMApiPo activity information about the invoice and the time it was created.
  5. After an arbitrary delay to simulate waiting for the invoice to be paid, adds to the BAMApiInvoice activity the time the invoice was paid.
  6. Ends the BAMApiInvoice activity.
The Main method of the MainApp class initializes the application. It does the following:
  1. Creates a DirectEventStream object.
  2. Starts several threads, and calls the Run method of the POApplication object in each thread.
  3. Starts several threads, and calls the Run method of the Shipment Application object in each thread.
  4. Starts several threads, and calls the Run method of the Invoice Application object in each thread.
The Global class defines constants that are used by the sample application, such as the number of threads to create and the percentage of purchase orders to reject.
In addition to the Visual Studio solution, the sample also contains a Microsoft Excel file that defines the activities.
You can find this sample at <Samples Path>\BAM\BamApiSample.
The following table shows the files in this sample and describes their purpose. 
File
Description
BamApiSample.cs
Instrumented application.
BamApiSample.csproj
Instrumented application project.
BamApiSample.sln
Instrumented application solution.
BamApiSample.xls
BAM definition style sheet.
Cleanup.bat
Batch file to remove deployed sample files.
Input.txt
Sample interceptor configuration input.
InterceptorConfig.cs
Interceptor configuration code for API sample.
InterceptorConfig.csproj
Interceptor configuration project.
Invoice_config.xml
Invoice interceptor configuration.
Invoice_interceptor.bin
Serialized invoice interceptor.
PurchaseOrder_config.xml
PurchaseOrder interceptor configuration.
PurchaseOrder_interceptor.bin
Serialized PurchaseOrder interceptor.
Setup.bat
Batch file to deploy and enlist sample files.
Shipment_config.xml
Shipment interceptor configuration.
Shipment_interceptor.bin
Serialized shipment interceptor.

How Can I view results

Use the following procedure to run the BAM API sample and view the results.

To run the BAM API sample

1.      Open a command prompt and run <Samples Path>\BAM\ BamApiSample\setup.bat. If you are using Windows 7 or Windows Server 2008 SP2, open the command prompt as administrator.
2.      Start Visual Studio, and open the <Samples Path>\BAM\ BamApiSample\BamApiSample.sln solution. If you are using Windows 7 or Windows Server 2008 SP2, start Visual Studio as administrator.
Important
The line //#define Interceptor in the BamApiSample.cs file must be commented out. Do not remove the “//” from this line. The BAM API sample uses only the code that is not inside an#if Interceptor preprocessor directive.
3.      Build the solution.
4.      Run <Samples Path>\BAM\BamApiSample\bin\debug\BamApiSample.exe.
The output will resemble the following:
New Purchase Order #17 Received
8 was shipped as pkg#87
New Purchase Order #18 Received.
Shipping package pkg#87 via DHL
13 was Approved.
18 was Rejected.
17 was Rejected.
11 was shipped as pkg#114
16 wsas Rejected.
Shipping package pkg#114 via DHL
Invoice #5 send for {2 5 1 9 4 8 11 }
Package pkg#49 was delivered
New Purchase Order #19 Received.
5.      After a minute or so, press CTRL+C or close the Command Prompt window to stop the BamApiSample program.

To view the results of running the BAM API sample

1.      Open SQL Server Management Studio.
2.      In SQL Server Management Studio, expand the server, expand Databases, expand BAMPrimaryImport, and then expand Tables.
3.      Right-click dbo.bam_BAMApiInvoice_Active and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_BAMApiInvoice_Active table are displayed in the right pane. Each row in the table represents a BAMApiInvoice activity that has been started, but has not been completed.
4.      Right-click dbo.bam_BAMApiPo_Completed and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_BAMApiPo_Completed table are displayed in the right pane. Each row in the table represents a BAMApiPo activity that has been completed.

BAM End-to-End

The End-to-End sample demonstrates how to correlate events from multiple components (in this case, three orchestrations and a pipeline) by using BAM.
BAM reconstructs the business activity that spans the pipeline component and orchestrations. At the lowest level, this works by calls to EventStream.EnableContinuation from each implementation component that expects more events for the activity. The call to Enable Continuation is explicit, while calls in Orchestration1 and Orchestration2 are made by adding a Continuation folder to the Tracking profile in one schedule, and a Continuation ID folder to the schedule that follows it.
The following diagram illustrates the workflow used in the sample.

Bam Flow

The BAM end-to-end flow shows how you can use BAM to gather information from a pipeline and multiple orchestrations and update a single activity.

How This Flow Was Designed and Why

The BAM end-to-end sample was designed to illustrate the following activities:
  • Using BAM within a pipeline.
  • Using the Tracking Profile Editor (TPE) to map activity items to shapes in an orchestration and elements of a message.
  • Using continuations to keep an activity active when multiple pieces of a solution contribute to the activity.
The sample works as follows:
  1. An input message is retrieved from the <Samples Path>\BamEndToEnd\Input folder.
  2. The pipeline component assigns a unique Document ID to the message, and uses the BAM API to begin a new BAM activity. The Document ID is attached as a separate part of the input message to make it available to the orchestrations.
  3. The service Orchestration1 is activated when the input message is received.
  4. Orchestration1 modifies the input message and passes it as a parameter to Orchestration2.
  5. Orchestration2 modifies the input message and sends it to the Message Box database, which activates Orchestration3.
  6. Orchestration3 modifies the message and writes it to the folder <Samples Path>\BamEndToEnd\Output.
  7. Each orchestration updates activity items in the BAM activity.
You can find this sample at <Samples Path>\BAM\BamEndToEnd.

To build and initialize this sample

1.      Open a command prompt and run <Samples Path>\BAM\BAMEndToEnd\Setup.bat. If you are using Windows 7 or Windows Server 2008 SP2, open the command prompt as administrator. Setup.bat builds and initializes the BAM infrastructure for this sample. Keep the command prompt open.
2.      Create a tracking profile to map Orchestration1, Orchestration2, and Orchestration3 to the BAM activity. (Because creating the tracking profile is a complex process, the detailed instructions are in a separate procedure called To create a tracking profile. This procedure appears later in this document.)
3.      Deploy the tracking profile BamEndToEnd.btt that you created in the previous step. In the command prompt change to the <Samples Path>\BAM\BamEndToEnd directory. To deploy the tracking profile, type the following line, and then press Enter:
“<BizTalkInstallationPath>\Tracking\bttdeploy” BamEndToEnd.btt
Important
You can ignore the message that the Continuation ID Orch1_ does not have a matching Continuation. This message is expected, because the continuation named Orch1_ is defined in the pipeline component, and not in the tracking profile.

To run this sample

·         Copy the file <Samples Path>\BamEndToEnd\InputMessage.xml into the folder <Samples Path>\BamEndToEnd\Input. After a few seconds, the message disappears from the Input folder, and an output message appears in the <Samples Path>\BamEndToEnd\Output folder.

To view the BAM data

1.      Open SQL Server Management Studio.
2.      In SQL Server Management Studio, expand the server, expand Databases, expand BAMPrimaryImport, and then expand Tables.
3.      Right-click dbo.bam_EndToEndActivity_Completed and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_EndToEndActivity_Completed table are displayed in the right pane. Each row in the table represents an EndToEndActivity activity that has been completed.

To re-run this sample

1.      Open a command prompt and change to the <Samples Path>\BAM\BamEndToEnd directory. If you are using Windows 7 or Windows Server 2008 SP2, open the command prompt as administrator. Type the following line:
“C:\Program Files\Microsoft BizTalk Server 2013\Tracking\bttdeploy” BamEndToEnd.btt /remove
Note
If you did not install BizTalk Server to the C drive, replace "C" with the letter of the drive where you installed BizTalk Server.
2.      Run <Samples Path>\BAM\BAMEndToEnd\Cleanup.bat. Cleanup.bat removes the BAM infrastructure for this sample.
3.      Perform the steps in to build and initialize this sample section in this topic.

To create a tracking profile

1.      Click Start, point to All Programs, point to Microsoft BizTalk Server 2013, and then click Tracking Profile Editor. If you are using Windows 7 or Windows Server 2008 SP2, right-click Tracking Profile Editor and then click Run as administrator.
2.      In the left pane of the Tracking Profile Editor window, click Click here to import a BAM Activity Definition.
3.      In the BAM Activity Definition Name section of the Import BAM Activity Definition dialog box, select EndToEndActivity, and then click OK.
4.      In the right pane of the Tracking Profile Editor window, click here to select an event source.
5.      In the Assembly Name section of the Select Event Source Parent Assembly dialog box, select Microsoft.Samples.BizTalk.BamEndToEnd.Services, and then click Next.
6.      In the Orchestration Name section of the Select Orchestration dialog box, select BamEndToEnd.Services.Orchestration1, and then click OK.
7.      In the left pane of the Tracking Profile Editor window, right-click EndToEndActivity, and then click New Continuation ID. Name the new continuation ID Orch1_. Repeat this step to create two more continuation IDs named Orch2_ and Orch3_.
8.      Right-click EndToEndActivity and then click New Continuation. Name the new continuation Orch2_. Repeat this step to create another continuation named Orch3_.
9.      Right-click the Receive1shapes, and then clicksContext Property Schemas.
10.  Scroll to the end of the Context Property Name list, and then double-click BAMEndToEnd.Services.PropertySchema.DocumentID.
11.  Expand <Schema>, and then drag Document ID in the right pane to Orch1_ in the left pane.
12.  Click the folder icon with the arrow ( ) twice to display the orchestration.
13.  Drag the Receive1 shape in the right pane to SBegin1 in the left pane.
14.  Drag the StartOrchestration_1 shape in the right pane to SEnd1 in the left pane.
15.  Right-click the StartOrchestration_1shapes, and then clicksMessage Payload Schemas.
16.  Double-click the row that contains the value “Message_2” in the Message column and the value “Message Body” in the Part column.
17.  Expand Schema2, and then drag Data2 in the right pane to Data1 in the left pane.
18.  Click Select Event Source, and then click Select Context Property.
19.  Scroll to the end of the Context Property Name list, and then double-click BAMEndToEnd.Services.PropertySchema.DocumentID.
20.  Expand <Schema>, and then drag Document ID to the Orch2_ continuation in the left pane.
Note
Do not confuse the Orch2_ continuation with the Orch2_ continuation ID. The icon that represents a continuation ID contains a key ( ), whereas the icon that represents a continuation does not contain a key ( ).
21.  Click Select Event Source, and then click Select Orchestration Schedule.
22.  In the Assembly Name section of the Select Event Source Parent Assembly dialog box, select Microsoft.Samples.BizTalk.BamEndToEnd.Services, and then click Next.
23.  In the Orchestration Name section of the Select Orchestration dialog box, select BamEndToEnd.Services.Orchestration2, and then click OK.
24.  Right-click the ConstructMessage_1shapes, and then clicksMessage Payload Schemas.
25.  Double-click the row that contains the value “Message_3” in the Message column and the value “BAM Part” in the Part column.
26.  Expand BAM Part, and then drag Document ID in the right pane to the Orch2_ continuation ID in the left pane.
Note
Do not confuse the Orch2_ continuation with the Orch2_ continuation ID. The icon that represents a continuation ID contains a key ( ), whereas the icon that represents a continuation does not contain a key ( ).
27.  Click the folder icon with the arrow ( ) twice to display the orchestration.
28.  Drag the ConstructMessage_1 shape in the right pane to SBegin2 in the left pane.
29.  Drag the Send_1 shape in the right pane to SEnd2 in the left pane.
30.  Right-click the Send_1shapes, and then clicksMessage Payload Schemas.
31.  Double-click the row that contains the value “Message_3” in the Message column and the value “Message Body” in the Part column.
32.  Expand Schema3, and then drag Data3 in the right pane to Data2 in the left pane.
33.  From the drop-down list above the right pane, select Message Payload Schemas.
34.  Double-click the row that contains the value “Message_3” in the Message column and the value “BAM Part” in the Part column.
35.  Expand BAM Part, and then drag Document ID in the right pane to the Orch3_ continuation in the left pane.
Note
Do not confuse the Orch3_ continuation with the Orch3_ continuation ID. The icon that represents a continuation ID contains a key ( ), whereas the icon that represents a continuation does not contain a key ( ).
36.  Click Select Event Source, and then click Select Orchestration Schedule.
37.  In the Assembly Name section of the Select Event Source Parent Assembly dialog box, select Microsoft.Samples.BizTalk.BamEndToEnd.Services, and then click Next.
38.  In the Orchestration Name section of the Select Orchestration dialog box, select BamEndToEnd.Services.Orchestration3, and then click OK.
39.  Right-click the Receive1shapes, and then clickMessage Payload Schemas.
40.  Double-click the row that contains the value “Message_3” in the Message column and the value “BAM Part” in the Part column.
41.  Expand BAM Part, and then drag Document ID in the right pane to the Orch3_ continuation ID in the left pane.
Note
Do not confuse the Orch3_ continuation with the Orch3_ continuation ID. The icon that represents a continuation ID contains a key ( ), whereas the icon that represents a continuation does not contain a key ( ).
42.  Click the folder icon with the arrow ( ) twice to display the orchestration.
43.  Drag the Receive1 shape in the right pane to SBegin3 in the left pane.
44.  Drag the Send_1 shape in the right pane to SEnd3 in the left pane.
45.  Right-click the Send_1shapes, and then clicksMessage Payload Schema.
46.  Expand Schema3, and then drag Data3 in the right pane to Data3 in the left pane.
47.  Right-click Document ID below the Orch2_ continuation, and then click Set Port Mappings.
Note
Do not confuse the Orch2_ continuation with the Orch2_ continuation ID. The icon that represents a continuation ID contains a key ( ), whereas the icon that represents a continuation does not contain a key ( ).
48.  In the Select Ports section of the Select Ports dialog box, click BamEndToEnd_ReceivePort, click the greater-than sign (>), and then click OK.
49.  Save the tracking profile to <Samples Path>\BAM\BamEndToEnd\BamEndToEnd.btt.

BAM API from an Orchestration Expression

You can find this sample at <Samples Path>\BAM\BamFromExpression.
The following table lists the files in this sample and describes their purpose. 
File
Description
BamDefinition.xls
BAM definition style sheet.
BamDefinition.xml
BAM definition.
BamFromExpression.btproj
Visual Studio tracking file project.
BamFromExpression.sln
Visual Studio solution.
Cleanup.bat
Batch file to undeploy the sample.
InputMessage.xml
Input message.
Orchestration1.odx
Orchestration.
PoSchema.xsd
Purchase order schema.
PropertySchema.xsd
Property schema.
Setup.bat
Batch file to compile and deploy the sample.
QueryBam.sql
SQL script.
Use the following procedures to create the tracking profile, run the sample, and view the results.

To create the tracking profile

1.      Open a command prompt and run <Samples Path>\BAM\BAMFromExpression\Setup.bat. If you are using Windows 7 or Windows Server 2008 SP2, open the command prompt as administrator. Setup.bat initializes the BAM infrastructure for this sample, and deploys the BAM activity.
2.      Click Start, point to All Programs, point to Microsoft BizTalk Server 2013, and then click Tracking Profile Editor. If you are using Windows 7 or Windows Server 2008 SP2, right-click Tracking Profile Editor and then click run as administrator.
3.      In the left pane of the Tracking Profile Editor window, click Click here to import a BAM Activity Definition.
4.      In the BAM Activity Definition Name section of the Import BAM Activity Definition dialog box, select FromExpressionPo, and then click OK.
5.      In the right pane of the Tracking Profile Editor window, click Click here to select an event source.
6.      In the Assembly Name section of the Select Event Source Parent Assembly dialog box, select Microsoft.Samples.BizTalk.BamFromExpression, and then click next.
7.      In the Orchestration Name section of the Select Orchestration dialog box, select BamFromExpression.Orchestration1, and then click OK.
8.      Right-click the Receive_1shapes, and then clicksMessage Payload Schema.
9.      Expand <Schema>, expand PurchaseOrder, expand from, and then drag PoID in the right pane to ActivityID in the left pane.
10.  Drag the following elements from the right pane, and drop them onto the named nodes in the left pane: 
From
To
Name
From
State
State
City
City
Phone
Phone
Total
PoTotal
11.  Click the folder icon with the arrow ( ) to display the orchestration.
12.  Drag the Receive_1 shape in the right pane to Received in the left pane.
13.  Drag the Send_1 shape in the right pane to send in the left pane.
14.  Save the tracking profile to <Samples Path>\BAM\BamFromExpression\ BamFromExpression.btt.
15.  On the Tools menu, click Apply Tracking Profile. .

To run this sample

·         Copy the file <Samples Path>\BamFromExpression\InputMessage.xml to <Samples Path>\BamFromExpression\Input.
In about 10 seconds the output message will appear in <Samples Path>\BamFromExpression\Output.

To view the BAM data

1.      Open SQL Server Management Studio.
2.      In SQL Server Management Studio, expand the server, expand Databases, expand BAMPrimaryImport, and then expand Tables.
3.      Right-click dbo.bam_FromExpressionPo_Completed, and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_FromExpressionPo_Completed table are displayed in the right pane. The one row, which has an activity ID of 123, represents the purchase order for $345 that was contained in the input message.
4.      Right-click dbo.bam_FromExpressionPoItem_Completed, and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_FromExpressionPoItem_Completed table are displayed in the right pane. The two rows, which have activity IDs 123_0 and 123_1, represent the items in the purchase order: Flash MC and Infrared Decoder.
5.      Right-click dbo.bam_FromExpressionPoItem_CompletedRelationships, and then click Open Table. If you are using SQL Server 2008 SP1, click Select top 1000 rows.
The contents of the bam_FromExpressionPoItem_CompletedRelationships table are displayed in the right pane. Each row in the table represents a relationship between a FromExpressionPoItem activity and a FromExpressionPo activity. The value in the Activity ID column refers to the activity ID of the FromExpressionPoItem activity. The value in the Reference Data column refers to the activity ID of the FromExpressionPo activity. In this case, the two records indicate that the Flash MC and Infrared Decoder items are associated with the purchase order for $345.

To re-run

1.      Open a command prompt and run <Samples Path>\BAM\BamFromExpression\Cleanup.bat to remove the tracking profile and other BAM infrastructure. If you are using Windows 7 or Windows Server 2008 SP2, open the command prompt as administrator.
2.      Run <Samples Path>\BAM\BamFromExpression\Setup.bat to compile the sample and deploy it.

Comments

Popular posts from this blog

Unit Testing In Biztalk 2009