Python SDK (Boto3) 指南

1. 概述

RustFS 与 S3 兼容，并支持 Boto3 SDK。

本指南涵盖

• 存储桶创建/删除
• 对象上传/下载/删除
• 列出对象
• 生成预签名 URL
• 大文件分块上传

---

2. 环境准备

2.1 示例配置

假设 RustFS 的部署如下：

Endpoint: http://192.168.1.100:9000
AccessKey: rustfsadmin
SecretKey: rustfssecret

2.2 安装 Boto3

我们推荐使用虚拟环境

python3 -m venv venv
source venv/bin/activate
pip install boto3

Boto3 依赖于 botocore，它将自动安装。

---

3. 连接到 RustFS

import boto3
from botocore.client import Config

s3 = boto3.client(
 's3',
 endpoint_url='http://192.168.1.100:9000',
 aws_access_key_id='rustfsadmin',
 aws_secret_access_key='rustfssecret',
 config=Config(signature_version='s3v4'),
 region_name='us-east-1'
)

✅ endpoint_url：指向 RustFS ✅ signature_version='s3v4'：RustFS 支持 v4 签名 ✅ region_name：RustFS 不验证区域；您可以使用任何值。

---

4. 基本操作

4.1 创建存储桶

bucket_name = 'my-bucket'

try:
 s3.create_bucket(Bucket=bucket_name)
 print(f'Bucket {bucket_name} created.')
except s3.exceptions.BucketAlreadyOwnedByYou:
 print(f'Bucket {bucket_name} already exists.')

---

4.2 上传文件

s3.upload_file('hello.txt', bucket_name, 'hello.txt')
print('File uploaded.')

---

4.3 下载文件

s3.download_file(bucket_name, 'hello.txt', 'hello-downloaded.txt')
print('File downloaded.')

---

4.4 列出对象

response = s3.list_objects_v2(Bucket=bucket_name)
for obj in response.get('Contents', []):
 print(f"- {obj['Key']} ({obj['Size']} bytes)")

---

4.5 删除对象和存储桶

s3.delete_object(Bucket=bucket_name, Key='hello.txt')
print('Object deleted.')

s3.delete_bucket(Bucket=bucket_name)
print('Bucket deleted.')

---

5. 高级功能

5.1 生成预签名 URL

5.1.1 下载链接 (GET)

url = s3.generate_presigned_url(
 ClientMethod='get_object',
 Params={'Bucket': bucket_name, 'Key': 'hello.txt'},
 ExpiresIn=600 # 10 minutes validity
)

print('Presigned GET URL:', url)

5.1.2 上传链接 (PUT)

url = s3.generate_presigned_url(
 ClientMethod='put_object',
 Params={'Bucket': bucket_name, 'Key': 'upload-by-url.txt'},
 ExpiresIn=600
)

print('Presigned PUT URL:', url)

您可以使用 curl 工具进行上传

curl -X PUT --upload-file hello.txt "http://..."

---

5.2 分块上传

适用于大于 10 MB 的文件，允许手动控制每个分块。

import os

file_path = 'largefile.bin'
key = 'largefile.bin'
part_size = 5 * 1024 * 1024 # 5 MB

# 1. Start upload
response = s3.create_multipart_upload(Bucket=bucket_name, Key=key)
upload_id = response['UploadId']
parts = []

try:
 with open(file_path, 'rb') as f:
 part_number = 1
 while True:
 data = f.read(part_size)
 if not data:
 break

 part = s3.upload_part(
 Bucket=bucket_name,
 Key=key,
 PartNumber=part_number,
 UploadId=upload_id,
 Body=data
 )

 parts.append({'ETag': part['ETag'], 'PartNumber': part_number})
 print(f'Uploaded part {part_number}')
 part_number += 1

 # 2. Complete upload
 s3.complete_multipart_upload(
 Bucket=bucket_name,
 Key=key,
 UploadId=upload_id,
 MultipartUpload={'Parts': parts}
 )
 print('Multipart upload complete.')

except Exception as e:
 # Abort upload
 s3.abort_multipart_upload(Bucket=bucket_name, Key=key, UploadId=upload_id)
 print('Multipart upload aborted due to error:', e)

---

6. 常见问题排查

问题	原因	解决方案
SignatureDoesNotMatch	未使用的 v4 签名	设置 signature_version='s3v4'
EndpointConnectionError	RustFS 地址错误或服务未启动	检查 endpoint 和 RustFS 服务状态
AccessDenied	凭证错误或权限不足	检查 AccessKey/SecretKey 或存储桶策略
PermanentRedirect	路径风格未启用	Boto3 默认使用虚拟主机，RustFS 只支持路径风格，但设置 endpoint 可以绕过

---

7. 附录：快速上传/下载脚本模板

def upload_file(local_path, bucket, object_key):
 s3.upload_file(local_path, bucket, object_key)
 print(f"Uploaded {local_path} to s3://{bucket}/{object_key}")

def download_file(bucket, object_key, local_path):
 s3.download_file(bucket, object_key, local_path)
 print(f"Downloaded s3://{bucket}/{object_key} to {local_path}")