DIY Exercise 11-1: Write DataWeave transformations

Time estimate: 3 hours

Objectives

In this exercise, you combine multiple data sources using DataWeave. You will:

·       Use a map function to iterate through arrays.

·       Use if else conditional statements to change output based on conditions.

·       Format numbers and dates.

·       Use sort and filter functions to modify data structures.

·       Use the read function to translate MIME types.

·       Create custom DataWeave modules to reuse and share DataWeave code

Scenario

You have been asked to enhance the flights Mule application to return the transactions and account details for each flight. You need to retrieve the results from the three online airline flights data sources (American, Delta, and United) and then join these results with results from two additional online data sources:

·       A Transactions web service that returns a list of all transactions related to a particular flight:
http://apdev-accounts-ws.cloudhub.io/api/transactions?wsdl

·       An Accounts API that returns a list of account details associated with flight transactions: http://apdev-accounts-ws.cloudhub.io/api/accounts

The Mule application should return a list of flight information along with transaction information and account details for each flight, organized into a collection of structured objects. A sample final output can be found in /files/module11/sample_output.xml (in the MUFundamentals4.x DIY Files zip that you can download from the Course Resources).

Import the starting project

Import the /files/module11/flights-mod11-dataweave-starter.jar deployable archive file (in the MUFundamentals4.x DIY Files zip that you can download from the Course Resources) into Anypoint Studio.

Note: This is an enhanced version of the flights Mule application solution from Module 11 of the Fundamentals course and has an updated RAML file.

Retrieve transactions

Create a new flow that makes a call to the Transactions web service. The web service takes a list of flight IDs in the SOAP request, then returns a list of transactions associated with each flight id. Here is the configuration information for the web service:

webservice.wsdl = http://apdev-accounts-ws.cloudhub.io/api/transactions?wsdl

webservice.service = TransactionServiceService

webservice.port = TransactionServicePort

webservice.address = http://apdev-accounts-ws.cloudhub.io/api/transactions

webservice.operation = GetTransactionsforFlights

Here is a sample request to the web service where each flightID in the request XML is composed by combining the airlineName, flightCode, and departureDate keys from each flight response:

<?xml version='1.0' encoding='UTF-8'?>

<ns0:GetTransactionsforFlights xmlns:ns0="http://training.mulesoft.com/">

  <flightID>UNITEDER38SD2015-03-20</flightID>

  <flightID>UNITEDER39RK2015-09-11</flightID>

  <flightID>UNITEDER39RJ2015-02-12</flightID>

</ns0:GetTransactionsforFlights>

Here is a sample response from the web service:

<?xml version='1.0' encoding='UTF-8'?>

<ns2:GetTransactionsforFlightsResponse xmlns:ns2="http://training.mulesoft.com/">

  <transaction>

    <amount>3177.0</amount>

    <customerRef>4456</customerRef>

    <flightID>UNITEDER39RK2015-09-11</flightID>

    <region>US</region>

    <transactionID>181948797</transactionID>

  </transaction>

  <transaction>

    <amount>6032.0</amount>

    <customerRef>4968</customerRef>

    <flightID>UNITEDER38SD2015-03-20</flightID>

    <region>US</region>

    <transactionID>181948717</transactionID>

  </transaction>

</ns2:GetTransactionsforFlightsResponse>

Store the transactions results in a variable so that the Transactions web service response can be combined with the flights response.

Retrieve account data for each flight transaction

Create a flow to make a request to the Accounts API to retrieve account details for each transaction record. The API has a POST api/accountList endpoint that expects a JSON-formatted array of account ids formatted as strings and returns a string that represents an array of Account objects in JSON format.

 

Here is the configuration information for the API:

accounts.host = apdev-accounts-ws.cloudhub.io

accounts.port = 80

accounts.basePath = /api

To make a request to the Accounts API, you must provide a valid input to the POST method and set the Requester-API header. The customerRef fields from the previous transactions SOAP API provide valid accountID values for the Accounts API requests. For example, you can write DataWeave expressions to extract these customerRef fields from the previous transactions response example:

[

  "4456",

  "4968"

]

Note: For ease of implementation, the Accounts API is not RESTful due to the POST api/accountList endpoint. The POST method is used to make it easy to include the array of account ids in the request rather than asking you to serialize the array with escaped characters into a GET request.

After the Accounts API returns a valid response, save the response payload to a flow variable so it can be joined with the flights and transactions responses.

Hint: To transform a string into a target object, you can use the DataWeave function:

read(data, "application/json")

Note: For an additional challenge, set the HTTP Request's output target to this variable without modifying the current event payload. 

Join all data into a nested object structure

The final step is to join flights, transactions, and accounts data. The Mule application contains starter code for the DataWeave transformation needed to build up this complex data structure. The essential transformation is located in the Combine all payloads and convert to XML Transform Message component in the getFlightsFlow.

 

Complete the join in two steps. First, join each flight object with the transaction with the same flightID value, so it creates a nested object:

airlineName: "UNITED",

flightCode: "ER39RK",

departureDate:" 2015-Mar-20",

...

transactions:

{

  idCustomer" 4564,

  transID: "181948544",

  amount: "735.0",

  ...

}

Next, join the accounts details (account name and account type) with each transaction:

airlineName: "UNITED",

flightCode: "ER39RK",

departureDate:" 2015-Mar-20",

...

  transactions:

    {

      idCustomer" 4564,

      transID: "181948544",

      amount: "735.0",

      nameCustomer: "Max Plank",

      type: "business"

      ...

    }

Call getFlightsFlow from the corresponding API interface flow

Inside the get:/flightstrans:mua-flights-transactions-api-config flow, add a Flow Reference component and set the flow name to getFlightsFlow.

Output combined data as XML data

Add additional DataWeave code to transform the output to valid XML.  Move or add some of the response data fields as XML attributes instead of XML elements as needed.  Here is an example final XML response:

<?xml version='1.0' encoding='UTF-8'?>

<flights>

    <flight flightCode="ER38sd">

        <airline>United</airline>

        <departureDate>2015-Mar-20</departureDate>

        <emptySeats>0</emptySeats>

        <fromAirportCode>MUA</fromAirportCode>

        <planeType>Boeing 737</planeType>

        <price>400.0</price>

        <toAirportCode>SFO</toAirportCode>

        <transactions>

            <transaction transID="181948544" amount="735.0">

                <idCustomer>4564</idCustomer>

                <nameCustomer>Max Plack</nameCustomer>

                <type>business</type>

                <amountUSFormatted>735.00</amountUSFormatted>

            </transaction>

            <transaction transID="181948717" amount="6032.0">

                <idCustomer>4968</idCustomer>

                <nameCustomer>Nick The Shoemaker</nameCustomer>

                <type>personal</type>

                <amountUSFormatted>6,032.00</amountUSFormatted>

            </transaction>

        </transactions>

    </flight>

</flights>

Create a reusable function to format the transaction amount

In the final DataWeave transformation, extract the logic used to format the transaction amount into a function. Save the function in a file named Formats.dwl in src/main/resources/modules and reference the module inside the final DataWeave transformation.

Verify your solution

Load the solution /files/module11/flights-mod11-dataweave-solution.jar (in the MUFundamentals4.x DIY Files zip that you can download from the Course Resources) and compare your solution.



Did you complete the exercise?

  Yes, I completed the exercise

  No, I did not complete the exercise

  I completed part of the exercise


Comments and/or feedback