Signiant Support

Media Shuttle Metadata Developer's Guide Print


Overview
Prerequisites
Flow Diagram
Media Shuttle Sample Web Form
Getting started
Handling Form Requests
Handling Form Submissions
Request Signing
REST API
Appendix A: Example of How to Generate a Signed Request URL
Appendix B: Code Examples
Appendix C: Media Shuttle Metadata Sample Form
Appendix D: Media Shuttle Metadata Signature Generator


Overview

The Signiant Media Shuttle Metadata feature enables you to link web forms with files that are uploaded using Signiant Media Shuttle.

Benefits

  • Improved traceability of submitted files: enables you to create delivery requests with pre-populated metadata and provide your suppliers with a unique URL which includes your ‘house ID’.
  • Improved tracking of submitted files: query Signiant Media Shuttle for the metadata associated with any historical delivery by providing the unique tracking code.
  • Quicker time to production: use your existing business logic and workflows associated with your existing web forms.
  • Lower cost of development: re-use your existing web infrastructure and simply link it to Signiant Media Shuttle.

Uses

  • Convert existing slow (FTP or HTTP) web delivery methods using Media Asset Managers and/or Digital Asset Management systems to high-speed delivery systems.
  • Create new web delivery methods that use your existing business logic to process deliveries without requiring the “heavy lifting” associated with integrating acceleration technologies into your web infrastructure.

Prerequisites

To take advantage of the Signiant Media Shuttle Metadata feature, ensure you have the following:
  1. An existing web server that can host a customizable web form application (serving form requests and responding to form submissions) that can be accessed securely by Media Shuttle (using https:// not http://).
  2. The ability to generate security signatures to exchange with Media Shuttle.
  3. A Media Shuttle submit portal configured for metadata.
  4. Your web server must have an SSL certificate installed issued by a trusted certificate authority such as Comodo. Self-signed certificates are not accepted by Media Shuttle.

Flow Diagram

This diagram illustrates the interaction between Media Shuttle and your web server.


Media Shuttle Sample Web Form

Media Shuttle provides a sample web form for each portal that allows you to see how this metadata feature is presented to the user.
 
To view the sample web form use: https://<portalUrlPrefix>.mediashuttle.com/metadata/sample/form

Getting started

The following steps explain how to create and register a metadata web form.

Step 1: Create a web form

The first step in creating a metadata enabled portal is building a customized web form that collects metadata for Media Shuttle portal submissions. This form is a standard HTML document hosted on your web server and is accessed securely by Media Shuttle (using https:// not http://).  
 
See Handling Form Requests and Handling Form Submissions for details on the general requirements and guidelines that must be followed to ensure your form is properly displayed and processed by Media Shuttle.

Step 2: Register your form with a portal

The second step is registering your web form with your portal using the Media Shuttle Management Console. During registration, Media Shuttle will assign a registration key that you will use to secure communications between your Media Shuttle portal and your web server.
 
To register your form, do the following:
  1. Log in to the Media Shuttle Management Console.
  2. Select Metadata.
  3. To use metadata collection, enable Enable metadata collection.
  4. In Metadata provider URL, type the secure URL for the web server that hosts your form.
  5. The Registration key is generated by Media Shuttle when you register a URL. (This key is used to sign requests as detailed in Request Signing.)
  6. Click Save Changes.
Note: Your registration key should be kept secure at all times. Do not share this key with anyone. Do not include it in JavaScript client-side code or embed it in a mobile client. If you suspect your registration key has been compromised, you can reset it using the Media Shuttle Management Console.

Handling Form Requests

Your web form application must be hosted on its own server and made available using HTTPS. To view our sample form, use the following link https://<portalUrlPrefix>.mediashuttle.com/metadata/sample/form.
 
Media Shuttle will request your web form using the URL you registered each time a portal member uploads a set of files by posting a request to your web server with the following post body form parameters:
  • packageId The unique Media Shuttle package identifier for the set of files being submitted to Media Shuttle. This ID can be used later to make REST API calls for package information including files and metadata.
  • redirectUrl The URL endpoint provided by Media Shuttle where your form must redirect to upon successfully processing the form.
  • metadataId Optionally use this to track individual form requests by passing a metadataId parameter in request links to your portal.
For example, you can give your portal users a link such as https://my-submit-portal.mediashuttle.com?metadataId=123 and Media Shuttle passes the key value pair metadataId=123 back to you in the web form post request. You must generate this metadataId and your web form application must perform any validation checks that you require.
 
The POST request is encoded using the standard web form application/x-www-form-urlencoded content type.
 
Upon receipt of the form request from Media Shuttle, your application should respond with an HTML document containing your form. Media Shuttle displays the form inside a sandboxed iframe within the portal browser window (this iframe has privileges to submit forms and execute JavaScript).
 
<iframe sandbox="allow-forms allow-scripts">
   your HTML document
</iframe>
 
Note Sending an X-Frame-Options response header value of DENY or SAMEORIGIN in form request responses causes your form to be displayed incorrectly in Media Shuttle. You can optionally send a value of ALLOW_FROM URI where the URI is the Media Shuttle portal URI or omit this header in your response.
 
Note Your HTML document is loaded as a self-contained web document and must include the scripts and styles it needs for your required presentation and functionality.

Handling Form Submissions

Your web form application must handle valid and invalid form submissions. Any invalid submission should be handled by returning the appropriate HTML document response (for example, the validation error redisplays the form with invalid field markers).
 
Upon successfully processing a web form submission, your server must respond with an HTTP 307 (Temporary Redirect) status. Your application may store the metadata itself. Media Shuttle also stores the metadata and continues with the transfer.
 
Your server response must specify a redirect target URL in the Location response header. This URL must be a properly signed URL that follows the signing process described in the Request Signing section.  This URL should be derived using the following request elements:
  • CanonicalURL should be formed using the redirectURL post parameter passed by Media Shuttle when the form was requested.
  • CanonicalQueryString must include the X-Sig-Algorithm and X-Sig-Date query string parameters.
  • RequestPayload must be formed using the same form body that was posted to your web server.
Here is an example form request:
 
POST https://your-web-form-uri
content-type: application/x-www-form-urlencoded
 
metadataId=123&packageId=X30G1zUlIThVdyGRbb&redirectUrl=https%3A%2F%2Fmy-submit-portal.mediashuttle.com%2Fmetadata%2Fv3.0%2Fmy-submit-portal%2Fpackage%2FX30G1zUlIThVdyGRbb%2Fmetadata
 
On success this form request should generate the following response:
 
HTTP/1.1 307 Temporary Redirect
Location: https://my-submit-portal.mediashuttle.com/metadata/v3.0/my-submit-portal/package/X30G1zUlIThVdyGRbb/metadata?X-Sig-Algorithm=SIG1-HMAC-SHA256&X-Sig-Date=<RequestDate>&X-Sig-Signature=<Signature>


Request Signing

All metadata related web requests sent to Media Shuttle must be signed in order to ensure that communications are secure. This communication includes:
  • The web form redirect request your server makes to Media Shuttle.
  • The REST API endpoint used to retrieve package details.
Media Shuttle also applies the same signing process when sending web form requests to your server so that your server can verify Media Shuttle's identity prior to serving the web form. Requests are signed using the registration key that was generated when your portal was configured to collect metadata.
 
Note: Your registration key should be kept secure at all times. Do not share this secret with anyone. Do not include it in JavaScript client-side code or embed it a mobile client. You can reset your registration key to a new value at any time using the Media Shuttle Management Console.
 
The signing process helps secure requests in the following ways:
  • Verifying the identity of the requester: signing ensures that the request has been issued by an entity that has access to the private registration key.
  • Protecting data in transit: certain key request elements are used in the calculation of the signature hash in order to prevent data tampering.
  • Protecting against potential replay attacks: a timestamp component is also included in the calculation of the signature hash to ensure requests cannot be replayed indefinitely. Requests are valid for 24 hours.
To sign a request, do the following:

Step 1: Create the canonical request

To begin the signing process, you need to generate the string that includes information from your request in a standardized (canonical) format. Formatting the request into an unambiguous canonical format before signing ensures that when the request is received, the receiving end can calculate the same signature for verification purposes.

The canonical request is defined as follows:
 
CanonicalRequest =
   CanonicalURL + '\n' +
   CanonicalQueryString + '\n' +
   HexEncode(Hash(RequestPayLoad))
 
The CanonicalURL is the normalized request URL from the beginning of the URL up to, but not including, the question mark character (?) that begins the query string parameters (if any).
 
The CanonicalQueryString should be constructed as follows:
  • URL-encode each parameter name and value according to the following rules:
    • Do not URL-encode any of the unreserved characters that RFC 3986 defines: A-Z, a-z, 0-9, hyphen (-), underscore (_), period (.) and tilde (~).
    • Percent-encode all other characters with %XY, where X and Y are hexadecimal characters (0-9 and uppercase A-F).
  • Sort the encoded parameter names by character code (in strict ASCII order).
  • Build the CanonicalQueryString by starting with the first parameter name in the sort list.
  • For each parameter, append the URL-encoded parameter name, followed by the character '=', followed by the URL-encoded parameter value. Use an empty string for parameters that have no value.
  • Append the character '&' after each parameter value except for the last value in the list.
  • The CanonicalQueryString must include parameters for the request and the two following query string parameters needed as part of the signing process:
    • X-Sig-Algorithm=SIG1-HMAC-SHA256 this parameter specifies the version of the signing algorithm used. Here we specify the algorithm as SIG1-HMAC-SHA256.
    • X-Sig-Date=<ISO8601 timestamp> this parameter specifies the ISO8601 formatted timestamp of the request. Requests older than one day are rejected by Media Shuttle.
HexEncode represents a function that returns the base-16 encoding of the supplied parameter in lowercase characters.
 
Hash represents a function that returns a SHA256 message digest.
 
The RequestPayload is the request body represented as an array of bytes in UTF-8 format.

Step 2: Create the string to sign

The string to sign includes the RequestDate plus the CanonicalRequest generated in Step 1.

The RequestDate is the ISO8601 timestamp of the request. It must match the timestamp specified by the X-Sig-Date query string parameter (explained in Step 1).
 
StringToSign =
   RequestDate + '\n' +
   CanonicalRequest

Step 3: Calculate the signature
 
To calculate the signature, first derive your signing key from your private registration key:
 
DerivedSigningKey = HMAC_SHA256(RegistrationKey, RequestDate)
 
HMAC_SHA256(key, data) represents an HMAC-SHA256 function that returns output in binary format. 
 
The RegistrationKey is the registration key that was generated when your portal was configured to collect metadata.
 
The RequestDate is the ISO8601 timestamp of the request. It must match the timestamp specified by the X-Sig-Date query string parameter.
 
Next, use the DerivedSigningKey generated above to sign the string generated in Step 2:
 
Signature = HexEncode(HMAC_SHA256(DerivedSigningKey, StringToSign))
 
After you've calculated the signature as a digest, the value is converted to its hexadecimal representation.

Step 4: Add the signature elements to the request

Append the hexadecimal representation of the signature to the request URL query string:
 
RequestURL = RequestURL + &X-Sig-Algorithm=SIG1-HMAC-SHA256&X-Sig-Date=<RequestDate>&X-Sig-Signature=<Signature>
 

REST API

You can query the details for the specified portal package including the list of files/folders, package size, status, sender and any associated metadata using the following REST API endpoint. 
 
GET /metadata/v3.0/portal/{portalUrlPrefix}/package/{packageId}
 
https://{portalUrlPrefix}.mediashuttle.com/metadata/v3.0/portal/{portalurlprefix}/package/{packageId}?X-Sig-Algorithm=SIG1-HMAC-SHA256&X-Sig-Date={RequestDate}&X-Sig-Signature={Signature}
 
Packages can be queried from the REST API immediately after form submission. You do not have to wait for the file upload to complete.

Path Parameters

  • portalUrlPrefix: the URL prefix portal of the portal's full URL. For example, for the portal submit-portal.mediashuttle.com, the value of this parameter is submit-portal.
  • packageId: the unique Media Shuttle package identifier. The packageId is posted to your web server in the web form request body as a form parameter. See the Handling Form Requests section for more information on obtaining the packageId.

Query String Parameters

The request query string must contain the following signature related parameters. See the Request Signing section for more details on how to set these parameters.
  • X-Sig-Algorithm: Identifies the signature algorithm used to sign the request URL. The parameter value should be set to SIG1-HMAC-SHA256
  • X-Sig-Date: the ISO8601 formatted request timestamp.
  • X-Sig-Signature: the request signature.

Response

Success Response

{
    "result": "SUCCESS",
    "packageDetails": 
    {
        "portalId": "578e1f3a-bccd-4ac3-b780-6f064003d6b0",
        "packageId": "3TnjFY7eUQa8MzfoTSDlGK",
        "creationDate": "2014-12-12T00:03:38.404+0000",
        "fileSize": 896403,
        "fileCount": 1,
        "status": "AVAILABLE",
        "sender": "sender@company.com",
        "deleted": false,
        "metadata": 
        {
            "title": "title",
            "description": "description"
        },
        "files": [{
            "rangeKey": "3TnjFY7eUQa8MzfoTSDlGK:a263cafd-950c-4b4c-882c-e52b458cb7e9",
            "portalId": "578e1f3a-bccd-4ac3-b780-6f064003d6b0",
            "packageId": "3TnjFY7eUQa8MzfoTSDlGK",
            "fileId": "a263cafd-950c-4b4c-882c-e52b458cb7e9",
            "isDirectory": false,
            "fileName": "file1",
            "fileSize": 896403,
            "uploadTime": "2014-12-12T00:03:38.584+0000"
        }]
    }
}


Error Response

{
    "result": "FAILURE",
    "resultCode": "<result code>"
}


The following error codes can be returned:
  • portal.package.not.found
  • portal.package.deleted
  • exception.getting.portal.package

Appendix A: Example of How to Generate a Signed Request URL

The following example shows how to generate a signed request URL for:
 
Portal: submit-portal.mediashuttle.com
Package ID: 4eMv
Registration Key: 2e751ce9-5684-4925-9cc3-0665802ebc55
 
To obtain the package details perform a GET request on the following REST API endpoint:
 
https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv
 

Step 1: Create a Canonical Request

First, generate the canonical request which concatenates the following components from each step into a single string:
  1. Start with the canonical URL followed by a newline character:  
https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv\n
 
  1. Add the canonical query string, followed by a newline character. The canonical query string should include any query string parameters required to invoke the endpoint plus the X-Sig-Algorithm and X-Sig-Date parameters required for signature verification. Since this endpoint does not require any query string parameters outside of the signature parameters, the canonical query string should include values for X-Sig-Algorithm and X-Sig-Date only in proper sort order. X-Sig-Algorithm should be set to the value SIG1-HMAC-SHA256. X-Sig-Date should be set to the current time formatted as a valid ISO 8601 timestamp. For the timestamp 2015-01-20T01:07:18.763Z, the canonical query string is:
 
X-Sig-Algorithm%3DSIG1-HMAC-SHA256&X-Sig-Date%3D2015-01-20T01%3A07%3A18.763Z
 
  1. Generate an SHA-256 hash of the request payload and encode the result as a lowercase hexadecimal string. Since the request payload is empty in this example, use the empty string as the input to the hash function. This results in the following hexadecimal string:
 
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 
The resulting canonical request string after completing these steps is:
 
https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv
X-Sig-Algorithm%3DSIG1-HMAC-SHA256&X-Sig-Date%3D2015-01-20T01%3A07%3A18.763Z
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 

Step 2: Create a String to Sign

Next, create a string to sign by taking the ISO 8601 formatted timestamp generated above, followed by a newline character, followed by the canonical request you constructed in Step 1:
 
2015-01-20T01:07:18.763Z
https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv
X-Sig-Algorithm%3DSIG1-HMAC-SHA256&X-Sig-Date%3D2015-01-20T01%3A07%3A18.763Z
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 

Step 3: Calculate the Signature

To calculate the request signature, you use a HMAC-SHA256 hashing function twice. Once to generate a signing key, and a second time to generate the final signature.
 
  1. Generate the signing key using the registration key assigned to your portal as the HMAC key and the ISO 8601 formatted request timestamp as the HMAC message to be hashed. Using the registration key 2e751ce9-5684-4925-9cc3-0665802ebc55 and the timestamp 2015-01-20T01:07:18.763Z, invoke the HMAC-SHA256 hashing function as follows:
HMAC_SHA256("2e751ce9-5684-4925-9cc3-0665802ebc55", "2015-01-20T01:07:18.763Z")
This invocation generates the following signing key:
 
ebf870730d4d914fd8c24761433524171e948cd851830e785343b5f9d0d0f56a

Note: this is the hexadecimal representation of the binary key. The key itself should be used in binary format.
  1. Calculate the signature using the derived signing binary key above (Step 3.1) as the HMAC secret key and the string to sign (Step 2) as the HMAC message to be hashed:
HMAC(DerivedSigningKey, StringToSign)

After calculating the signature as a digest, convert the binary value to its hexadecimal representation.

This should result in the following final signature value:
 
139319aec19208168aaea515d0110b75d36c73de852c3265fc9758834d1b78ec

For a Linux example see Generating a Signature using Linux.

Step 4: Add the Signing Information to the Request

After you calculate the signature, you add the required signing elements to the request query string:
 
https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv?X-Sig-Algorithm=SIG1-HMAC-SHA256&X-Sig-Date=2015-01-20T01%3A07%3A18.763Z
&X-Sig-Signature=139319aec19208168aaea515d0110b75d36c73de852c3265fc9758834d1b78ec

Appendix B: Code Examples

Generating a HMAC-SHA256 using Java

The following Java code demonstrates generating an HMAC using the HMAC-SHA256 algorithm:

 
static byte[] HmacSHA256(byte[] key, String data) throws Exception {
      String algorithm="HmacSHA256";
      Mac mac = Mac.getInstance(algorithm);
      mac.init(new SecretKeySpec(key, algorithm));
      return mac.doFinal(data.getBytes("UTF8")); 
}

Generating a SHA256 Message Digest using Java

The following Java code demonstrates generating an SHA-256 message digest:
 
public static String generateSHA256Hash(String payload)
        throws NoSuchAlgorithmException, UnsupportedEncodingException {
 
    MessageDigest md = MessageDigest.getInstance(SHA256_ALGORITHM);
    md.update(payload.getBytes("UTF-8"));
    byte[] bytes = md.digest();
 
    StringBuilder builder = new StringBuilder();
    for (int i = 0; i < bytes.length; i++) {
        builder.append(Integer.toString((bytes[i] & 0xff) + 0x100, 16).substring(1));
    }
    return builder.toString();
}

Generating a Signature using Linux Command Line

Use the following Linux command to generate a signature as detailed in step 3.2 on page :
 
> head -c -1 <FileWithStringToSign> | openssl dgst -sha256 -mac HMAC -macopt hexkey:ebf870730d4d914fd8c24761433524171e948cd851830e785343b5f9d0d0f56a

Sample PHP Script to Process Web Form Post and Redirect to Media Shuttle

<?php
 
$registrationKey   = "registration key";
$entityPayload     = file_get_contents('php://input');
$entityPayloadHash = hash("sha256", $entityPayload);
 
foreach (explode("&", $entityPayload) as $chunk) {
    $param = explode("=", $chunk);
    if ($param[0] === "redirectUrl") {
        $redirectUrl = urldecode($param[1]);
    }
}
 
$timestamp      = date(DATE_ISO8601);
$algorithmParam = "X-Sig-Algorithm=SIG1-HMAC-SHA256";
$dateParam      = "X-Sig-Date=" . $timestamp;
$queryString    = urlencode($algorithmParam) . "&" .
                  urlencode($dateParam);
$stringToSign   = $timestamp   . "\n" .
                  $redirectUrl . "\n" .
                  $queryString . "\n" .
                  $entityPayloadHash;
$signingKey     = hash_hmac("sha256", $timestamp, $registrationKey, true);
$signature      = hash_hmac("sha256", $stringToSign, $signingKey);
$signatureParam = "X-Sig-Signature=" . $signature;
$redirectUrl    = $redirectUrl . "?" .
                  $algorithmParam . "&" .
                  $dateParam . "&" .
                  $signatureParam;
header("Location: " . $redirectUrl, TRUE, 307);
 
?>
 

Sample Visual Basic Script that generates a URL that returns metadata from Media Shuttle

Imports Rework '3rd Party Library: https://www.nuget.org/packages/Rework/
Imports System.Web
Imports System.Text
Imports System.Text.RegularExpressions

Public Class Signiant
    Public Shared Function GenerateSignedUrl(requestUrl, requestBody, registrationKey) As String
        Dim requestTimestamp = DateTime.UtcNow.ToString("o")

        ' Sample Credentials
        requestUrl = "https://submit-portal.mediashuttle.com/metadata/v3.0/portal/submit-portal/package/4eMv"
        requestBody = ""
        registrationKey = "2e751ce9-5684-4925-9cc3-0665802ebc55"
        requestTimestamp = "2015-01-20T01:07:18.763Z"

        ' Generate canonical query String
        Dim algorithmParam = "X-Sig-Algorithm=SIG1-HMAC-SHA256"
        Dim dateParam = "X-Sig-Date=" + requestTimestamp
        Dim canonicalQueryString = HttpUtility.UrlEncode(algorithmParam) + "&" + HttpUtility.UrlEncode(dateParam)
        canonicalQueryString = Regex.Replace(canonicalQueryString, "(%[0-9a-f][0-9a-f])", Function(c) c.Value.ToUpper())

        ' Generate the string to sign
        Dim requestBodyHash = Crypto.ToSHA(requestBody, Crypto.SHA_Type.SHA256).ToLower()
        Dim stringToSign As String = requestTimestamp + ChrW(10) + requestUrl + ChrW(10) + canonicalQueryString + ChrW(10) + requestBodyHash

        ' Generate the signing key
        Dim signingKey As Byte()
        Using HMACSHA256 As New System.Security.Cryptography.HMACSHA256(Encoding.UTF8.GetBytes(registrationKey))
            signingKey = HMACSHA256.ComputeHash(Encoding.UTF8.GetBytes(requestTimestamp))
        End Using

        ' Generate request signature
        Dim signature As Byte()
        Using HMACSHA256 As New System.Security.Cryptography.HMACSHA256(signingKey)
            signature = HMACSHA256.ComputeHash(Encoding.UTF8.GetBytes(stringToSign))
        End Using
        Dim signatureHex = Replace(BitConverter.ToString(signature), "-", "").ToLower()

        ' Generate the signed URL
        Dim signatureParam = "X-Sig-Signature=" + signatureHex
        Dim signedUrl = requestUrl + "?" + algorithmParam + "&" + dateParam + "&" + signatureParam

        Return signedUrl
    End Function
End Class

Appendix C: Media Shuttle Metadata Sample Form

Use the Media Shuttle Metadata Sample Form to configure a metadata collection form for Media Shuttle using Webtask. Refer to the instructions in Readme.md for additional information on using Webtask and the samples.

Appendix D: Media Shuttle Metadata Signature Generator

Use the Media Shuttle Metadata Signature Generator sample code and instructions to generate signed request URLs for Media Shuttle metadata requests.

Attachments

SigniantMediaShuttleMetadata.pdf SigniantMediaShuttleMetadata.pdf