Speed Traps in the Cloud: The Real Cost of Distance in Storage Performance

When we set out to benchmark cloud storage competitors against the incumbent (AWS), we expected to find some performance differences—but we didn’t anticipate just how dramatically location could make or break your storage strategy. What started as a straightforward comparison of AWS S3, Backblaze B2, Wasabi Hot Storage, and Storj Object Storage quickly revealed a sobering reality: the wrong storage decision can cripple your global operations before you even realize what’s happening. Picture this—your Sydney team trying to access files stored in your EU bucket experiences a staggering 95% performance drop compared to local access, turning what should be routine file operations into productivity nightmares. Over the course of comprehensive testing across four global regions with varying file sizes and time periods, we uncovered performance patterns that could save—or cost—your organization significant time, money, and sanity. Whether you’re planning a global expansion, optimizing existing operations, or simply trying to avoid the architectural quicksand that comes with poor storage choices, this data-driven analysis reveals the hidden speed traps lurking in your cloud storage decisions.

Objective

This test aims to benchmark the performance of Storj Object Storage, AWS S3, Backblaze B2, and Wasabi Hot Storage. The primary goal is to quantify the performance differences among these services across various regions and parameters, including node location, bucket location, file size, and time of day. Additionally, the test seeks to determine how cloud storage performance can be expected to vary based on its location and access point for a production environment with globally distributed resources.

Test Setup

Initial/Comprehensive Test Suite

• Compute:
  • • HostHatch was used to create compute nodes in Los Angeles, London, New York, and Sydney.
    • Each node was configured to run an hourly cron job to test the performance of every S3 Compliant bucket, for each provider, in each region, with various file sizes.

• S3 Compliant Storage:

• • • Wasabi:
      • ap-southeast-2
      • eu-west-1
      • us-east-1
      • us-west-1
    • AWS: AWS S3 Buckets were created in each of the following regions with the standard storage class. Glacier Instant access was also considered, but not used.
      • ap-southeast-2
      • eu-west-2
      • us-east-1
      • us-west-1
    • Backblaze: Backblaze B2 Buckets were created in each of the following regions. B2 Overdrive was also considered, but access to B2 Overdrive requires a multi-petabyte commitment.
      •  EU Central
      • US East
      • US West
    • Storj:
      • Global
      • US Select

• Benchmark Software:

• • Warp S3 Benchmarking Tool was was used to run tests. YAML files were configured for each S3 provider and bucket region combination for GET and PUT tests with variables for concurrency of requests, file size, and length of test.

Time of Day Testing

A second test was run with a similar set up. HostHatch nodes were set up in the same cities. S3 Buckets were also set up in the same regions, and Warp was used on an hourly basis via a cron job from each node to test the performance of the S3 compliant services. The key difference was that each compute node only accessed its same-region bucket for each competitor and only using a small file size/concurrency to stick to a maintainable bandwidth over the course of a full week.

Test Procedure

Initial/Comprehensive Test Suite

Every hour on the hour a script was run that would use Warp to benchmark the performance of every bucket from every compute node. For each bucket:

• First the script would clear the bucket and run a PUT test for 200MB files of random data. (Concurrency 10, 30s)
• Then it would run a GET test with those files from the previous PUT test. (Concurrency 10, 30s)
• Next it would clear the bucket and run a PUT test for 1GB files of random data. (Concurrency 10, 1m)
• Then another GET test with those same 1GB files. (Concurrency 10, 1m)
• Finally, the bucket would be cleared once more so that a PUT test could be run with 5GB files. (Concurrency 4, 2m)
• That would be followed by another GET test for those same 5GB files. (Concurrency 10, 2m)
• Once all of those tests were complete, the script would continue on to the next bucket, running all the same tests, until the suite had been run for every bucket from every provider.
• NOTE: The suite of tests often took longer than an hour to run, so whenever UTC Hour is referenced, that is the hour at which the test suite began, and does not necessarily reflect the time at which a specific bucket was benchmarked.

Time of Day Testing

Every hour on the hour a script was run that would use Warp to benchmark the performance of each competitor within each region. For each competitor, the nearest bucket was accessed/tested from each region thus:

• First the script would clear the bucket and run a PUT test for 10MB files of random data. (Concurrency 8, 10s)
• Then it would run a GET test with those files from the previous PUT test. (Concurrency 8, 10s)
• A script would then run to delete the files in that bucket.
• Once those tests were complete, the script would continue on to the next bucket, running the same tests, until the suite had been run for each provider in the region.
• NOTE: For these tests, the hourly cron job was significantly lighter, and did complete close to the beginning of the hour.

Results

Every individual test had its full results saved in a .json.zst file. Warp was used to analyze the results of each test, and a script pulled the Average Throughput and Time to First Byte (TTFB) from all of the GET tests. It also pulled the Average Throughput from the PUT tests. Occasionally the tests on a particular bucket would encounter errors (not so for the Time of Day Testing). Some reasons why this might occur included, the files from the bucket being cleared by a test that was running concurrently from a different node (this happened infrequently) or the bucket hitting a downloading bandwidth cap (this happened for Backblaze US East). For any tests where the number of errors was greater than half the number of total requests, the results were dismissed as invalid. 

The full results can be found at in this spreadsheet: 

https://docs.google.com/spreadsheets/d/1bs-ox4of8Wh8bdVjGfSEThi8-8Rb0B2CFg5MPLDHJ1E/

Here are some aggregated summaries of the results split by Size of File:

200MB

1GB

5GB

These tests also sought to investigate the question of whether certain times of day would see performance decreases in each region. It stands to reason that many storage customers perform their backup operations in the middle of the night, and therefore, throughput might decrease within a region during the time that most customers in that region are performing backups. Less than one full day of testing was deemed to be less than sufficient to make claims about the effect of Time of Day on the performance of object storage providers; therefore, one full week of tests with a lower bandwidth requirement were conducted. The following pages present charts that show the performance of buckets from each provider when accessed by nodes within the same region.

Time Series

Los Angeles

London

New York

Sydney

Some times of day in these charts have sharp peaks, such as 1300 in Backblaze London or 1800 in Storj New York. These were primarily caused by instances where one test out of seven for that time of day had such extreme results that it skewed the data based on a possible outlier. A few outliers were removed from the data because upon further inspection, the results of Warp’s benchmark were nonsensical, but the frequency of outliers in Backblaze data made it seem like a real effect that couldn’t be ignored. Again, the complete data can be viewed at the following link:

https://docs.google.com/spreadsheets/d/1bs-ox4of8Wh8bdVjGfSEThi8-8Rb0B2CFg5MPLDHJ1E/edit?gid=326497009#gid=326497009

Conclusions and Analysis

Performance Variability Throughout the Day

Analysis of performance data across different time periods reveals that provider performance is not constant throughout the day, with some services showing significant temporal variations that could impact operational planning.

Key Findings:

• Wasabi demonstrates the most pronounced time-based performance fluctuations, with notable degradation during certain hours, likely correlating with increased backup and batch processing workloads from other customers
• Backblaze shows performance instability with extreme outliers that can dramatically skew average performance for specific time periods, making it less predictable for consistent workloads
• AWS demonstrates relatively stable performance throughout the day with minimal time-based variations
• Storj maintains consistent performance across different time periods with less volatility than other providers

Operational Implications:

While these patterns suggest potential optimization opportunities, establishing reliable performance windows for production workloads would require more extensive data collection over multiple weeks to account for day-of-week variations and seasonal patterns.

Global Performance Analysis for Distributed Teams

For organizations with globally distributed teams accessing shared storage, traditional cloud providers suffer from severe cross-regional performance penalties that can render services nearly unusable for remote team members.

Cross-Regional Performance Penalties

When teams access storage buckets located in distant regions, performance degradation is dramatic:

Example – Sydney team accessing EU-based storage (200MB files):

• AWS EU bucket from Sydney: 25.3 MiB/s GET throughput, 513.6 ms latency
• Storj Global from Sydney: 49.3 MiB/s GET throughput, 881.0 ms latency

This represents a 95% improvement in throughput for the remote team member, though Storj Global performs in the middle tier overall for Sydney access rather than leading the field.

File Size Performance Characteristics

Testing across different file sizes reveals distinct architectural behaviors that create specific competitive advantages for different workload types.

Large File Upload Optimization

Storj US Select demonstrates unique performance scaling behavior for upload operations. When accessing US Select from New York, PUT throughput actually improves with larger file sizes:

• 200MB files: 689.1 MiB/s PUT throughput
• 5GB files: 768.4 MiB/s PUT throughput (+11.5% improvement)

This contrasts sharply with traditional providers, where performance typically degrades with larger files:

• AWS US East: -20.5% throughput decrease from 200MB to 5GB
• Backblaze US East: -33.7% throughput decrease
• Wasabi US East: -23.4% throughput decrease

Provider-Specific Operational Considerations

Multi-Regional Deployment Complexity

Backblaze presents unique operational challenges for organizations requiring multi-regional storage deployments. Unlike other providers, Backblaze requires separate accounts for each region where buckets are established, significantly complicating administrative overhead and access management for globally distributed organizations.

Performance Predictability

Provider reliability ranking for consistent performance:

1. AWS: Most predictable, though with severe cross-regional penalties
2. Storj: Consistent baseline with minimal time-based variation
3. Wasabi: High peak performance but with time-based fluctuations
4. Backblaze: Strong average performance but prone to extreme outliers

Summary of Key Findings

This comprehensive benchmarking study of object storage providers reveals significant performance variations based on geographic location, file size, access patterns, and temporal factors. The testing methodology evaluated AWS S3, Backblaze B2, Wasabi Hot Storage, and Storj across multiple global regions using standardized workloads.

Geographic Performance Dependencies

Performance characteristics vary dramatically based on the relationship between storage bucket location and access point:

Regional Optimization Benefits: All providers demonstrate superior performance when accessed from geographically proximate locations. Traditional cloud providers show their strongest performance metrics when users access regionally-local buckets.

Cross-Regional Performance Penalties: Significant performance degradation occurs when accessing storage across continental boundaries. Download throughput can decrease by 80-95% when teams access distant regional buckets, with correspondingly higher latency.

Global vs Regional Architecture Trade-offs: Providers with global distribution models offer more consistent cross-regional performance, while regional providers excel in local access scenarios but suffer severe penalties for distant access.

File Size Performance Characteristics

Testing across 200MB, 1GB, and 5GB file sizes revealed distinct architectural behaviors:

Upload Performance Scaling: Some storage architectures demonstrate improved throughput efficiency with larger file sizes, while others show degraded performance. This creates specific advantages for different workload types, particularly for large file operations such as video processing, database backups, and scientific datasets.

Download Performance Consistency: GET operations generally show more stable performance across file sizes compared to PUT operations, though specific provider implementations vary significantly.

Temporal Performance Variations

Performance consistency throughout the day varies considerably among providers:

Time-Based Fluctuations: Some providers show pronounced performance variations during peak usage periods, likely correlating with increased backup and batch processing workloads from customer bases.

Predictability Factors: Providers range from highly consistent performance profiles to those with significant outliers that can impact operational planning and SLA compliance.

Operational Considerations

Administrative Complexity: Providers differ significantly in their multi-regional deployment requirements, with some requiring separate account management for each geographic region.

Performance Predictability: Organizations must balance peak performance capabilities against consistency requirements, particularly for globally distributed teams.

Implementation Guidance

The optimal storage solution depends on specific organizational requirements:

For Regionally Concentrated Operations: Traditional cloud providers offer excellent performance when teams and data remain within the same geographic region.

For Globally Distributed Teams: Organizations with international operations should prioritize solutions that maintain acceptable performance across regions rather than optimizing for peak local performance.

For Large File Workloads: Specific provider architectures demonstrate advantages for upload-intensive operations, particularly in certain geographic configurations.

For Predictable Operations: Organizations requiring consistent performance should evaluate time-based variations and select providers with stable performance profiles aligned to their operational requirements.

The benchmarking data demonstrates that no single provider excels across all metrics and use cases. Success requires matching specific architectural strengths to organizational workflows, geographic distribution, and performance requirements.

Integrated  Digital  Solutions  (IDS)  specializes  in  delivering  cutting-edge  technology  solutions  tailored  to  meet  diverse  business  needs.  Their  expertise  spans  video  streaming,  cloud  solutions,  software  engineering, data analytics, and AI & machine learning. IDS focuses on innovation, offering services like  content  management,  cloud  migration,  custom  software  development,  and  advanced  analytics  to  empower  businesses.  With  a  commitment  to  operational  agility,  security,  and  user-centric  design,  IDS  positions itself as a trusted partner for digital transformation and long-term success.