Dealing with Spaces in S3 Files Stored within Folders in a Bucket

Files and folders within a bucket can create problems when you access them. In this article, we will see some code to deal with this problem. To list all files in a given bucket in S3:

def self.all(bucket_name)
  Aws.config[:s3] = {
            region: 'us-west-1',
            retry_limit: 0

  s3 =
  resp = s3.list_objects(bucket: bucket_name)
  resp.contents.each do |object|
    puts "#{object.key}"

You can now run this in rails console to see all the files. If you have created folders inside your bucket, you need to modify the above program:

def self.all(bucket_name)
  Aws.config[:s3] = {'same as above'}

  s3 =
  # reference an existing bucket by name
  bucket = s3.bucket(bucket_name)

  # enumerate every object in a bucket
  bucket.objects(prefix: 'Audio Files  Book  1/').each do |obj|
    puts "#{obj.key}"

In this case I have a folder called Audio Files Book 1/ within the bucket name. This prints:

Audio Files  Book  1/
Audio Files  Book  1/Audio 1-1.mp3
Audio Files  Book  1/Audio 1-10.mp3
Audio Files  Book  1/Audio 1-11.mp3
Audio Files  Book  1/Audio 1-12.mp3
... more like this omitted here

Now can use the S3 API to copy these files to another folder without spaces in the file names.

def self.copy
  source_bucket_name = 'your-source-s3-bucket-name'
  target_bucket_name = 'your-target-s3-bucket-name'
  source_key = 'Audio Files  Book  1/Audio 1-1.mp3'
  target_key = 'audio/Audio1-1.mp3'

  s3 =
  arguments = {bucket: target_bucket_name, 
               copy_source: source_bucket_name + '/' + source_key, 
               key: target_key}

  puts "Copying file #{source_key} to #{target_key}."

The configure is a private method.


def self.configure
  Aws.config[:s3] = {region: 'us-west-1',
                       retry_limit: 0}

To move a list of files, we can iterate over the files and copy to the desired folder.

  def self.move_videos(bucket_name)
    # configure S3 credentials

    s3 =
    bucket = s3.bucket(bucket_name)
    bucket.objects.each do |object|
      key = object.key

      if key.start_with?('Video')
        clean_file_name = key.gsub(' ', '')
        copy(key, "video/#{clean_file_name}")
    puts 'Done'    

This code copies the file with a file name that does not contain any spaces. Spaces can cause problems when accessing the files. The versions of aws gems I am using in this project:

aws-eventstream (1.0.1)
aws-partitions (1.95.0)
aws-sdk-core (3.22.1)
aws-sdk-kms (1.6.0)
aws-sdk-s3 (1.16.0)
aws-ses (0.6.0)
aws-sigv4 (1.0.3)

You can download a file from S3 and save it on your local computer.

  def self.download_file
   #  configure S3 not shown

    s3 =

    bucket = s3.bucket('your-s3-bucket-name')

    bucket.objects.each do |object|
      key = object.key

      if key.start_with?('Video') && key.end_with?('avi')
        clean_file_name = key.gsub(' ', '')
        object.get(response_target: "./#{clean_file_name}")
        puts "Downloaded #{clean_file_name}"

    puts "Done"    

In this case I am downloading only video files that has avi extension. Installed ffmpeg to convert .avi file to .mov so that it can be played on a browser. You can do:

brew install ffmpeg

When opening the converted .mov file using Quicktime, it had issues. If you open the .avi file using Quicktime, it converts to .mov and this has no problem.


You need to be careful with the gem version because it can be confusing, since they use AWS or Aws and other nasty surprises that I encountered like these error messages:

NameError (uninitialized constant AWS::S3)
NoMethodError (undefined method `bucket' for #<Aws::S3::Client>)
ArgumentError (:bucket option must not contain a forward-slash (/))

I spent a few hours going over the AWS docs, source code of the gems and reading the tests to fix the issues. I hope this helps you a few hours of your time.


S3 File Move Github Source
S3 Docs
AWS S3 version 3 docs

Related Articles

Create your own user feedback survey