AWS S3: Get Node

The AWS S3: Get Node allows a workflow to retrieve a file and its metadata from an S3 bucket.

Node Properties

There are three main configuration sections for the AWS S3: Get Node ...

Credential Configuration

There are two options for configuring credentials:

• Choose a Service Credential: Retrieves your AWS credentials from the selected service credential.
  • Credential Name Template: Choose which of your application's AWS service credentials you would like to use. Note: Service credentials are not accessible from Edge Workflows, so only direct configuration is available for them. This field is templatable.
• Enter Connection Info: Configure the AWS credentials directly within the node. All fields are required.
  • AWS Region Template: The AWS region to connect to. This field is templatable.
  • AWS Access Key ID Template: The AWS access key ID associated with the AWS account you wish to use. This field is templatable.
  • AWS Secret Access Key Template: The AWS secret access key associated with the AWS account you wish to use. This field is templatable.

We recommend that you create an AWS Key/Secret pair specifically for use in the Workflow Engine - a key that has access to the proper AWS S3 buckets. We also recommend that you store these values as service credentials.

AWS Permissions

In order to perform a get operation, the AWS user must have, at minimum, the following IAM Policy:

{
  "Version": "YYYY-MM-DD",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::<bucket-name>/*"]
    }
  ]
}

File Configuration

Next is to configure the fields required by S3. All applicable fields are templatable:

• Bucket Name: (Required) The S3 bucket where your file is located.
• File Name: (Required) The name of the file to retrieve, including the extension. If the file is in a subdirectory within the bucket, the path to the file should be included in this field. (e.g. "path/to/my/file.ext")

Result Type

Next, choose what to return from the file request. Options are the following ...

• Get File Contents: (Default) If selected, the file contents will be placed on the payload under the result path defined below. Files retrieved by this method have a maximum size of 5MB.
• Get Download Link: If selected, a pre-signed URL to download the file will be placed on the payload under the result path defined below. This option is useful for larger files, as it does not have a size limitation. (Note: Download links retrieved this way will expire after 7 days.)
• Local File Path Template: In edge workflows running GEA version 2.0.0 and higher, you can save the retrieved file to the edge device's file system. If selected, specify a path to where the file should be saved within the agent's Docker container. The field supports string templates.

When selecting the Get File Contents or Local File Path Template options, you must also specify the following a File Encoding. The default is UTF-8.

When selecting the Local File Path Template option, you must also choose how to handle conflicts if the file already exists at the specified path:

• Cancel and return an error if the file exists: The operation will fail and return an error if a file already exists at the specified path.
• Append my content to the file if it exists: The fetched content will be appended to the end of the file if it already exists.
• Overwrite the file with my new content if it exists: (Default) The existing file will be overwritten with the newly fetched content.

Output

Finally, provide a Result Path as a payload path for where the result of the operation will be placed. This value is required.

When the operation successfully executes, the result will contain two keys: the value (whose contents depend on the selected result type as described in the examples), and some metadata about the file. When the the operation encounters an error, the result will contain a single error key.

Node Example

The AWS S3: Get Node's output varies with the selected result type:

When Get File Contents is selected, the result will look like the following:

{
  "value": "here are the file contents ...",
  "metadata": { ... }
}

When Get Download Link is selected, the result will look like the following:

{
  "value": "https://<region>.amazonaws.com/<bucket-name>/<key>",
  "metadata": { ... }
}

When Local File Path Template is selected, the result will look like the following:

{
  "value": "/path/to/saved/file.ext",
  "metadata": { ... }
}

Node Errors

In the case of an error, the resulting object will contain an error key instead of the normal value and metadata keys.

{
  "error": "Access Denied"
}

Related Nodes

• AWS S3: Put Node
• AWS Lambda Node