S3 Presigned URLs

Amazon S3 has been in the news lately:

Top Defense Contractor Left Sensitive Pentagon Files on Amazon Server With No Password

The RNC Files: Inside the Largest US Voter Data Leak

Cloud Leak: How A Verizon Partner Exposed Millions of Customer Accounts

S3’s default configuration does not allow public access to the contents of a bucket, but these stories all feature bucket or object permissions that were open to the world. It’s evident that it’s a common mistake, but how can we avoid it?

S3 presigned URLs are one answer. A single API call will provide a time-limited URL which will allow access to an object, even if it’s otherwise private. Here’s an example:

$ curl https://s3.us-east-2.amazonaws.com/uiuc-presigned-url-example/secret.txt

<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>24FF5CD84B7510CE</RequestId>
<HostId>Q29LqDuXtn8x+L+3ol1YbIhse+
2XJbUs1HxV3Eq2Fa3krwTPNhS6yu1Ffx8DHgBsrsehvCFeN6Q=</HostId>
</Error>

That file is private, so clicking on that link gives an a AccessDenied error. However, with the right access in the hosting account, I can use the AWS CLI to request a pre-signed link:

$ aws s3 presign s3://uiuc-presigned-url-example/secret.txt --expires-in 604800

https://s3.us-east-2.amazonaws.com/uiuc-presigned-url-example/secret.
txt?X-Amz-Date=20170720T182534Z&X-Amz-SignedHeaders=host&X-Amz-Creden
tial=ASIAIYLQNVRRFNZOCFBA%2F20170720%2Fus-east-2%2Fs3%2Faws4_request&
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=604800&X-Amz-Security-
Token=FQoDYXdzEJP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDOLWx95j90zPxGh7WSL
dAVnoYoKC4gjrrR1xbokFWRRwutmuAmOxaIVcQqOy%2Fqxy%2FXQt3Iz%2FohuEEmI7%2
FHPzShy%2BfgQtvfUeDaojrAx5q8fG9P1KuIfcedfkiU%2BCxpM2foyCGlXzoZuNlcF8o
hm%2BaM3wh4%2BxQ%2FpShLl18cKiKEiw0QF1UQGj%2FsiEqzoM81vOSUVWL9SpTTkVq8
EQHY1chYKBkBWt7eIQcxjTI2dQeYOohlrbnZ5Y1%2F1cxPgrbk6PkNFO3whAoliSjyRC8
e4TSjIY2j3V6d9fUy4%2Fp6nLZIf9wuERL7xW9PjE6eZbKOHnw8sF&X-Amz-Signature
=a14b3065ab822105e8d7892eb5dcc455ddd603c61e47520774a7289178af9ecc"

That returns a long URL which will work for one week from the time it was created.

$ curl "https://s3.us-east-2.amazonaws.com/uiuc-presigned-url-example/secret.txt?X-Amz-Date=20170720T182534Z&X-Amz-SignedHeaders=host&X-Amz-Credential=ASIAIYLQNVRRFNZOCFBA%2F20170720%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=604800&X-Amz-Security-Token=FQoDYXdzEJP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDOLWx95j90zPxGh7WSLdAVnoYoKC4gjrrR1xbokFWRRwutmuAmOxaIVcQqOy%2Fqxy%2FXQt3Iz%2FohuEEmI7%2FHPzShy%2BfgQtvfUeDaojrAx5q8fG9P1KuIfcedfkiU%2BCxpM2foyCGlXzoZuNlcF8ohm%2BaM3wh4%2BxQ%2FpShLl18cKiKEiw0QF1UQGj%2FsiEqzoM81vOSUVWL9SpTTkVq8EQHY1chYKBkBWt7eIQcxjTI2dQeYOohlrbnZ5Y1%2F1cxPgrbk6PkNFO3whAoliSjyRC8e4TSjIY2j3V6d9fUy4%2Fp6nLZIf9wuERL7xW9PjE6eZbKOHnw8sF&X-Amz-Signature=a14b3065ab822105e8d7892eb5dcc455ddd603c61e47520774a7289178af9ecc"

This is a secret file. Keep it safe.

Presigned URLs can be used to safely share files with collaborators or exclusively display content to logged-in, trusted users on a website. They can’t stop a trusted user from saving content and reposting it elsewhere, so consider the human factor when you’re using them.

Amazon is currently sending out alerts to account holders that have publicly-readable S3 buckets. If we receive a notice, we’ll forward it on to that account’s admin contacts for consideration.

 

Introducing Amazon EC2 G3 Instances

You can now launch G3 instances, the latest generation of Amazon EC2 Accelerated Compute Instances. G3 instances make it easy to procure a powerful combination of GPU, CPU, and host memory for workloads such as 3D rendering, 3D visualizations, graphics-intensive remote workstations, video encoding, and virtual reality applications.

Backed by the NVIDIA Tesla M60 GPUs, G3 instances offer double the CPU power per GPU, and double the host memory per GPU when compared to the most powerful GPU cloud instance available today. This allows you to do complex modeling and 3D visualization analyses such as medical image processing, computer-aided design, or seismic visualization jobs in much less time than possible with any other GPU cloud instance.

ou can launch G3 instances using the AWS Management Console, AWS CLI, AWS SDKs, and third-party libraries. G3 instances are available in three instance sizes in US East (Ohio), US East (N. Virginia), US West (Oregon), US West (N. California), AWS GovCloud (US), and EU (Ireland), with support for more regions coming soon. To learn more about G3 instances, visit the AWS Blog.

Launch a G3 Instance

Amazon Research Awards Seeking Proposals

Amazon has opened a call for proposals for the 2017 round of Amazon Research Awards (ARA) in a number of areas, including machine translation, natural language understanding, search, robotics, and more. The program is open to faculty members at academic institutions in North America and Europe and awards up to 80,000 USD in cash and 20,000 USD in AWS promotional credits.

Proposal submissions are accepted until September 15, 2017.

For complete information visit the Amazon Research Rewards webpage. Should you complete a proposal, please contact the Illinois AWS team and make them aware so they can provide support.

Free Amazon Web Services Summit

Do you have questions about how you can use Amazon Web Services (AWS) to enhance your research, storage, or website hosting? AWS will host a FREE seminar in Chicago on Wednesday July 26 and Thursday July 27 at the McCormick Place Lakeside Center. To register online or see additional details visit https://aws.amazon.com/summits/chicago/.

This summit is a great, low-cost way to attend technical sessions and workshops, bootcamp training events, and labs. AWS engineers, solutions architects and AWS partners will be present and available throughout the event.

Onsite registration begins at 7:30am on Wednesday followed by labs and the keynote presentation at 9:30am.

Amazon Web Services Technical Updates

Below are a number of technical updates announced by Amazon Web Services (AWS) recently:

    1. FPGA nodes in Amazon Web Services (AWS) are now available
    2. Research IT and Technology Services continue to work with the University on getting the Terms and Conditions agreed to and contracts in place for Microsoft Azure and Google Cloud platform. For announcements visit http://cloud.illinois.edu.
    3. A new version of the AWS Deep Learning machine image was released in April. It includes popular deep learning frameworks, including MXNet, Caffe, Caffe2, TensorFlow, Theano, CNTK, Torch and Keras. It also includes popular packages including Jupyter notebooks with Python 2.7 and Python 3.4 kernels, Matplotlib, Scikit-image, CppLint, Pylint, pandas, Graphviz, Bokeh Python packages, Boto and Boto 3 and the AWS CLI. The Deep Learning machine image also comes packaged with Anaconda 2 and Anaconda 3 Data Science platforms. More information can be found at https://aws.amazon.com/marketplace/pp/B01M0AXXQB
    4. The AWS Elastic Map Reduce (EMR) service, which provides tools such as Hadoop, Spark, Hbase, Presto and Hive, updated to versions of Presto (0.170), Apache Zeppelin (0.7.1), and Hue (3.12.0) on Amazon EMR release 5.5.0. Presto 0.170 includes support for LDAP authentication and various improvements and bug fixes. Hue 3.12.0 adds new features to the SQL editor, timeline and pivot graphing for visualization, and email notifications for Apache Oozie workflow completion. https://aws.amazon.com/about-aws/whats-new/2017/04/updates-to-presto-apache-zeppelin-apache-flink-and-hue-now-available-on-amazon-emr-release-5-5-0/