Docs

Azure Storage Container

MLADU will first connect to your Azure Storage Account and then to a Storage Container, based on the name that you provide, to access blobs and files for data transfers. We recommend storing your data in Azure Blobs for maximum file size capacity. The Microsoft Azure Account and Storage Container are created, owned, and governed by your organization. MLADU will not modify, administrate, or modify your storage container settings or contents. You will be in complete control of your storage container. We do recommend that you create a separate storage container for data transfers. This will avoid any collisions with files required for work in progress.

MLADU will connect to your Azure Storage Account through the use of an Account Connect String and key. We recommend rotating and providing MLADU with a new key each data transfer for enhanced access and data security. We also highly recommend that you rotate the provided key after successful completion and validation of a data transfer.

File Size Requirements

Azure Blob storage is a highly scalable and cost-effective option for storing unstructured data, such as images, scientific data, and documents. However, it is important to be aware of the following limits when using Blob storage:

  • Maximum Blob size: The maximum size for a single Blob in Azure Blob storage is 4 TB
  • Maximum number of Blobs per container: There is no hard limit to the number of Blobs that can be stored in a container, but there are performance implications for storing large numbers of Blobs in a single container
  • Maximum number of containers per storage account: A single storage account can contain up to 500 Tb of data, spread across an unlimited number of containers

Built-In Data Integrity

Azure storage does automatically add a content MD5 hash on smaller files (i.e., files uploaded that are small enough to not need blocking). Additionally, Azure allows you to set the content hash along with other properties when you upload a file.

MLADU will verify that the Azure copy of the blob/file is identical to the downloaded file by comparing Azure's server-generated content_hash to the downloaded file's content hash. Azure uses the MD5 algorithm to compute the hash of each block in a Azure blob with each block being 4 MB. To learn more, please read about file integrity for Azure storage.

Topics