Ceph Bytes by Octopus

Ceph Bytes by Octopus

How To Use REST API with RADOS Gateway

Moshe Levy 21 mins0

Introduction

Managing storage in Ceph RADOS Gateway (RGW) often requires sending HTTP requests with authentication headers, which can be difficult to do manually. To make this easier, we created a bash script that by using it make these requests more user friendly. In this post we will see How To Use REST API with RADOS Gateway to achieve lifecycle management and flexibility with Ceph.

This script is a first step toward building a self-service storage management app that will interact with Ceph in the background. It allows users to:

  • Manage buckets (create, list, modify, delete)
  • Set quotas (limit storage and objects per user or bucket)
  • Control access (generate/revoke keys, change policies)
  • Manage users (create, list, modify, delete)

Format REST API Operations In a Script

#!/bin/bash

# Usage : rgw_curl.sh $access_key $secret_key $rgw_host:$rgw_port $http_request $admin_resource $query_string   

#To format the output in a more readble form pipe the execution of the script >>>>>> | grep '<?xml' | xmllint --format - | grep '<Name>' | sed 's/<Name>//; s/<\/Name>//'

access_key=$1
secret_key=$2 
rgw_endpoint="$3"
http_request=$4
resource="$5"
query_string="$6"

contentType="application/x-compressed-tar"
dateTime=`date -R -u`

# If resource is 'bucket', use the S3 API instead of the Admin API
if [[ "$resource" == "bucket" ]]; then
    # Use S3 API: http://<RGW_HOST>:<RGW_PORT>/<bucket_name>
    bucket_name="$query_string"
    headerToSign="${http_request}

${contentType}
${dateTime}
/${bucket_name}"

    signature=`echo -en "$headerToSign" | openssl sha1 -hmac ${secret_key} -binary | base64`
    if [[ "$http_request" == "PUT" ]]; then
      echo "Creating S3 bucket: ${bucket_name} at http://${rgw_endpoint}/${bucket_name}"
    elif [[ "$http_request" == "GET" && "$query_string" == "" ]]; then
      echo "List the buckets at http://${rgw_endpoint}"
    elif [[ "$http_request" == "GET" && "$query_string" != "" ]]; then
      echo "List the bucket ${query_string} information at http://${rgw_endpoint}"
    elif [[ "$http_request" == "DELETE" ]]; then
      echo "DELETE the bucket ${query_String} from the gateway"
    fi

    curl -X ${http_request} -H "Content-Type: ${contentType}" -H "Date: ${dateTime}" \
        -H "Authorization: AWS ${access_key}:${signature}" -H "Host: ${rgw_endpoint}" \
        "http://${rgw_endpoint}/${bucket_name}" 

# Handle user quota operations via Ceph Admin API
elif [[ "$resource" == "quota" ]]; then
    headerToSign="${http_request}

${contentType}
${dateTime}
/admin/user"

    signature=`echo -en "$headerToSign" | openssl sha1 -hmac ${secret_key} -binary | base64`

    if [[ "$http_request" == "PUT" ]]; then
        echo "[INFO] Setting user quota: ${query_string}"
    elif [[ "$http_request" == "GET" ]]; then
        echo "[INFO] Fetching user quota for UID: ${query_string}"
    fi

    curl -X ${http_request} -H "Content-Type: ${contentType}" -H "Date: ${dateTime}" \
        -H "Authorization: AWS ${access_key}:${signature}" -H "Host: ${rgw_endpoint}" \
        "http://${rgw_endpoint}/admin/user?quota&${query_string}"

else
    # Use the default Admin API for other resources
    headerToSign="${http_request}

${contentType}
${dateTime}
/admin/${resource}"

    signature=`echo -en "$headerToSign" | openssl sha1 -hmac ${secret_key} -binary | base64`

    echo "Query URL: http://${rgw_endpoint}/admin/${resource}?${query_string}"

    curl -X ${http_request} -H "Content-Type: ${contentType}" -H "Date: ${dateTime}" \
        -H "Authorization: AWS ${access_key}:${signature}" -H "Host: ${rgw_endpoint}" \
        "http://${rgw_endpoint}/admin/${resource}?${query_string}"
fi

How To Use The Script To Interact with RADOS Gateway

User Related Actions

Create a new user with default permissions:

 bash rgw_curl.sh <access-key> <secret> <url:port> PUT user "uid=test444&display-name=user-test444"

Output:
Query URL: http://<url>/admin/user?uid=test444&display-name=user-test444
{"tenant":"","user_id":"test444","display_name":"user-test444","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"test444","access_key":"<access-key>","secret_key":"<access-key>"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

List the Users

 bash rgw_curl.sh <access-key> <secret> <url:port> GET user list

Output:
Query URL: http://<url>/admin/user?list
{"keys":["micahel23","Itay1","test444","test35","sysadmin","moshelevy1"],"truncated":false,"count":6}

Get a Specified User Information

bash rgw_curl.sh <access-key> <secret> <url:port> GET user "uid=moshelevy1"

Output:
Query URL: http://<url>/admin/user?uid=moshelevy1
{"tenant":"","user_id":"moshelevy1","display_name":"moshe-levy-1","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"moshelevy1","access_key":"FSNZ5U44175","secret_key":"IAjr5r35JxCobaoLAVYpL44kO1"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

Delete a User

bash rgw_curl.sh <access-key> <secret> <url:port> DELETE user "uid=moshelevy1"

Output:
Query URL: http://<url>/admin/user?uid=moshelevy1

Modify a User

bash rgw_curl.sh <access-key> <secret> <url:port> POST user "uid=test35&display-name=Updated-test35&email=newemail35@example.com&max-buckets=20"

Output:
Query URL: http://<url>/admin/user?uid=test35&display-name=Updated-test35&email=newemail35@example.com&max-buckets=20
{"tenant":"","user_id":"test35","display_name":"Updated-test35","email":"newemail35@example.com","suspended":0,"max_buckets":20,"subusers":[],"keys":[{"user":"test35","access_key":"<access-key>","secret_key":"<secret-key>"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"read"}],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

Generate a new Access Key/Secret Key

bash rgw_curl.sh <access-key> <secret> <url:port> POST user "uid=test123&generate-key=true"

Output:
Query URL: http://<url>/admin/user?uid=test35&generate-key=true
{"tenant":"","user_id":"test35","display_name":"Updated-test35","email":"newemail35@example.com","suspended":0,"max_buckets":20,"subusers":[],"keys":[{"user":"test35","access_key":"UI77B8IY8","secret_key":"mtBZDVa5577ceBwbk"},{"user":"test35","access_key":"YJ8C792MFW44T8OQ","secret_key":"7PkezOz9OeXo1qT9Dwzd3x456vsz2s"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"read"}],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

Add a Specific Access Key for a User

bash rgw_curl.sh <access-key> <secret> <url:port> POST user "uid=test35&access-key=13243546&secret-key=1324354657"

Output:
Query URL: http://<url:port>/admin/user?uid=test35&access-key=13243546&secret-key=1324354657
{"tenant":"","user_id":"test35","display_name":"Updated-test35","email":"newemail35@example.com","suspended":0,"max_buckets":20,"subusers":[],"keys":[{"user":"test35","access_key":"1324646","secret_key":"1324765767"},{"user":"test35","access_key":"UI371KQ7PYC1XPTB8IY8","secret_key":"mtBZDVatFA0567Cf765765eBwbk"},{"user":"test35","access_key":"YJ8C792MFWGNRE8ST8OQ","secret_key":"7PkezOz9OeXo1qT9Dwzd3xnbGV8FvWxCscBvsz2s"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"read"}],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

Remove an Access Key (Revoke Access)

bash rgw_curl.sh <access-key> <secret> <url:port> DELETE user "uid=test123&access-key=newAccessKey123"

Set User-Scoped Quotas (Limit Storage & Objects)

bash rgw_curl.sh <access-key> <secret> <url:port> PUT quota "uid=test123&quota-type=user&max-objects=5000&max-size=100G"


Output:
[INFO] Setting user quota: uid=test443&quota-type=user&max-objects=500&max-size=10737418240&enabled=true

Suspend a User (Disable Access)

bash rgw_curl.sh <access-key> <secret> <url:port> PUT user "uid=test123&suspended=true"

Output:
Query URL: http://<url>/admin/user?uid=test443&suspended=true
{"tenant":"","user_id":"test443","display_name":"test-user-443","email":"","suspended":1,"max_buckets":1000,"subusers":[],"keys":[{"user":"test443","access_key":"13243546576879","secret_key":"9786756453"},{"user":"test443","access_key":"136879","secret_key":"97ddd756453"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","system":false,"admin":false,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":true,"check_on_raw":false,"max_size":10737418240,"max_size_kb":10485760,"max_objects":500},"temp_url_keys":[],"type":"rgw","mfa_ids":[]}

List All User’s Buckets

bash rgw_curl.sh <access-key> <secret> <url:port> GET bucket "uid=test123"

Bucket Releated Actions

Set Bucket-Scoped Quotas (Limit Per Bucket)

bash rgw_curl.sh PUT quota "uid=test123&quota-scope=bucket&max-objects=1000&max-size=10G"

Create a New Bucket

bash rgw_curl.sh PUT bucket "bucket=my-bucket"

List All Bcukets

bash rgw_curl.sh GET bucket

Output:
List the buckets at <url:port>
<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>sysadmin</ID><DisplayName>System-Administrator</DisplayName></Owner><Buckets><Bucket><Name>newest_bucket</Name><CreationDate>2025-02-03T13:23:15.940Z</CreationDate></Bucket><Bucket><Name>newest_bucket122</Name><CreationDate>2025-02-03T13:23:24.982Z</CreationDate></Bucket><Bucket><Name>test-bucket</Name><CreationDate>2025-01-29T14:57:08.454Z</CreationDate></Bucket><Bucket><Name>test-bucket2</Name><CreationDate>2025-02-03T09:17:31.577Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>

Get General Information About a Specified Bucket

bash rgw_curl.sh GET bucket "test-bucket"

Output:
List the bucket test-bucket information at <url:port>
<?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>test-bucket</Name><Prefix></Prefix><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated><Marker></Marker></ListBucketResult>

Get Information About Bucket Access Control

bash rgw_curl.sh GET bucket "test-bucket?acl"

Output:
List the bucket test-bucket?acl information at <url:port>
<?xml version="1.0" encoding="UTF-8"?><AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>sysadmin</ID><DisplayName>System-Administrator</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>sysadmin</ID><DisplayName>System-Administrator</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>

Delete a Bucket

bash rgw_curl.sh DELETE bucket "bucket=my-bucket"

Empty (Purge) a Bucket Before Deletion

bash rgw_curl.sh DELETE bucket "bucket=my-bucket&purge-objects=true"

Set Bucket Quotas (Limit Size and Object count)

bash rgw_curl.sh POST quota "uid=test123&quota-scope=bucket&bucket=my-bucket&max-objects=1000&max-size=10G"

Assign a Bucket to a Specific User

bash rgw_curl.sh PUT bucket "bucket=my-bucket&uid=test123"

Output:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>MissingContentLength</Code><Message></Message><BucketName>bucket=test--bucket&amp;uid=test443</BucketName><RequestId>tx0000047bb4535b5d63-0067b602f3-2700a-zone</RequestId><HostId>2700a-site</HostId></Error>

Summary

This guide, How To Use REST API with RADOS Gateway, outlines a script and its examples that mark the first milestone in developing a self-service storage app backed by a Ceph cluster. By enabling remote access to core Ceph RADOS Gateway (RGW), the script streamlines lifecycle management, covering user and bucket operations, quota management, and access control—laying the groundwork for a scalable and user-friendly platform. Next steps include designing the routing API, planning the app architecture, and implementing strong authentication and authorization. Ultimately, this will evolve into a comprehensive storage management solution, empowering users with self-service capabilities while ensuring administrative control and data security.

If you don’t already have a Ceph cluster and a Ceph RADOS Gateway, refer to our guide: How to Deploy High Availability Ceph on Proxmox.


Moshe Levy

Moshe Levy is a DevOps Engineer with hands-on experience in Linux, Ansible, OpenShift, and Ceph Storage, specializing in automation, cloud-native technologies, and scalable infrastructure.