Migrating Files to Custom Storage

The Custom storage feature is available with the Plugin Expansion plan and higher.

If you're on the Plugin for Startup plan, consider upgrading to access this feature. For more details, visit our pricing page or contact support.

Connecting Custom storage changes where new uploads go. Files uploaded before the switch stay on Topol's storage, and the templates that use them keep pointing there. Until those URLs are updated, the images keep loading from Topol's storage and count toward your Topol data traffic.

This tutorial moves those older files to your own bucket and points your templates at them. It works the same way for every supported provider (AWS S3, Google Cloud Storage, Cloudflare R2 and DigitalOcean Spaces), in five steps:

1. Connect your own storage to the API Token.
2. Request an export of your files from Topol.
3. Upload the export to your bucket.
4. Update the image URLs in your templates.
5. Check that nothing points to Topol's storage anymore.

TIP

Custom storage is configured per API Token, and each API Token has its own storage on Topol's side. If your integration uses several API Tokens, go through this tutorial once for each of them.

Step 1: Connect your own storage

Connecting the storage comes before the export: nothing new is uploaded to Topol's storage after the switch, so the export in Step 2 contains every file. Set up your bucket and connect it to the API Token with the tutorial for your provider:

• Custom AWS S3 Storage
• Custom Google Cloud Storage
• Custom Cloudflare R2 Storage
• Custom DigitalOcean Spaces Storage

Follow the bucket, CORS and access settings in those tutorials exactly. Every image must be publicly readable, because email clients load images without any authentication.

After Finish setup, new uploads go to your bucket and the File Manager lists the contents of your bucket instead of Topol's storage.

WARNING

Between this step and Step 3, the File Manager shows only the files uploaded after the switch. Images in existing templates keep loading from Topol's storage in the meantime, so nothing breaks in your emails. Keep this gap short.

Step 2: Request the export

The export is a copy of everything on Topol's storage for one API Token. Contact our support to request it free of charge, and tell us which API Token it's for.

We send you a download link to a single plugin-assets folder with the same structure as your storage on Topol's side:

plugin-assets/
└── 12345/                  ← your Topol account ID
    ├── 1001/               ← one folder per user, named after their userId
    │   ├── banner.png
    │   ├── logo.png
    │   └── spring-sale/    ← a folder the user created in the File Manager
    │       └── hero.jpg
    ├── 1002/
    │   └── product.jpg
    └── 1003/
        └── ...

• The first level is your Topol account ID (12345 in the example).
• The second level holds one folder per user, named after the userId your integration passes in the authorize option (1001, 1002 and 1003 in the example).
• Inside each user folder are that user's files, including any folders they created in the File Manager.

The path of each file in the export matches the end of its URL. For example, this image in a template:

https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/plugin-assets/12345/1001/spring-sale/hero.jpg

is this file in the export:

plugin-assets/12345/1001/spring-sale/hero.jpg

That is why the structure must stay exactly as it is.

Step 3: Upload the export to your bucket

Upload the whole plugin-assets folder to the root of your bucket without renaming or moving anything. After the upload, the bucket should contain keys such as plugin-assets/12345/1001/banner.png. This is the same layout the Plugin uses for new uploads, so old and new files sit side by side.

The files need the same public access the Plugin gives to its own uploads. On AWS S3, Google Cloud Storage and DigitalOcean Spaces, that means a public-read ACL on every file. Cloudflare R2 has no per-file ACLs, and its files are public through the bucket's Public URL.

The commands below run from the directory that contains the downloaded plugin-assets folder. Replace my-company-assets with your bucket name.

AWS S3

aws s3 sync ./plugin-assets s3://my-company-assets/plugin-assets \
  --acl public-read

Google Cloud Storage

gcloud storage cp -r ./plugin-assets gs://my-company-assets \
  --predefined-acl=publicRead

Cloudflare R2

aws s3 sync ./plugin-assets s3://my-company-assets/plugin-assets \
  --endpoint-url https://<cloudflare-account-id>.r2.cloudflarestorage.com

DigitalOcean Spaces

aws s3 sync ./plugin-assets s3://my-company-assets/plugin-assets \
  --acl public-read \
  --endpoint-url https://fra1.digitaloceanspaces.com

For DigitalOcean Spaces, replace fra1 with your bucket's region. For a Cloudflare R2 bucket created under a jurisdiction, add it to the endpoint (for example, https://<cloudflare-account-id>.eu.r2.cloudflarestorage.com).

WARNING

A file uploaded without public access is stored correctly but shows up as a broken image in the editor and in every email that uses it.

Recreate the folders (AWS S3, Cloudflare R2 and DigitalOcean Spaces)

On these three providers, the File Manager shows a folder only when the bucket contains an empty folder object whose key ends with / (for example, plugin-assets/12345/1001/spring-sale/). The Plugin creates one whenever a user creates a folder, but aws s3 sync and similar tools upload files only. Without these objects, the files inside user-created folders still load in templates but can't be reached from the File Manager.

Run this once after the upload, from the same directory, to create a folder object for every folder below the user level:

find plugin-assets -mindepth 3 -type d | while read -r dir; do
  aws s3api put-object --bucket my-company-assets --key "$dir/"
done

For Cloudflare R2 and DigitalOcean Spaces, add the same --endpoint-url as in the upload command. Google Cloud Storage lists folders without these objects, so it needs no extra step.

Step 4: Update the image URLs in your templates

Templates are stored on your side, so the URL update happens in your database. Every image from Topol's storage has the same old prefix in front of plugin-assets/..., and the update replaces that prefix with your bucket's prefix while keeping the rest of the URL. Only files uploaded before the switch need this, since new uploads already get URLs pointing to your bucket.

Find the old prefix

The old prefix is everything before plugin-assets/ in an image URL from Topol's storage. It always has this form:

https://storage.googleapis.com/topol-io-plugin-<api-token-id>/

Open any template that uses an uploaded image and look at its src. In this one, the old prefix is https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/:

https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/plugin-assets/12345/1001/banner.png

Find the new prefix

The Plugin itself is the most reliable source. Upload any image through the File Manager after Step 1, insert it into a template, save it, and check the image's src. Everything before plugin-assets/ is the new prefix.

The new prefix depends on your provider and the values entered when connecting the storage:

Provider	New prefix	Example
AWS S3	https://s3.<region>.amazonaws.com/<bucket>/	https://s3.eu-central-1.amazonaws.com/my-company-assets/
AWS S3 with a CloudFront URL	<CloudFront URL>/	https://d1a2b3c4d5e6f7.cloudfront.net/
Google Cloud Storage	https://storage.googleapis.com/<bucket>/	https://storage.googleapis.com/my-company-assets/
Cloudflare R2	<Public URL>/	https://pub-0123456789abcdef.r2.dev/
DigitalOcean Spaces	<Public URL>/	https://my-company-assets.fra1.digitaloceanspaces.com/

Replace the prefix

With both prefixes known, each URL changes like this:

Before: https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/plugin-assets/12345/1001/banner.png
After:  https://s3.eu-central-1.amazonaws.com/my-company-assets/plugin-assets/12345/1001/banner.png

Images appear in more than one place in the template JSON (for example, src on image blocks and background-url on sections), so replace the prefix across the whole stored string rather than in selected fields:

// Before
{ "src": "https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/plugin-assets/12345/1001/banner.png" }
{ "background-url": "https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/plugin-assets/12345/1002/product.jpg" }

// After
{ "src": "https://s3.eu-central-1.amazonaws.com/my-company-assets/plugin-assets/12345/1001/banner.png" }
{ "background-url": "https://s3.eu-central-1.amazonaws.com/my-company-assets/plugin-assets/12345/1002/product.jpg" }

Update everything you store that contains template content:

• the template JSON, so the editor loads the new URLs
• the template HTML, including any copies your automations or scheduled campaigns send
• saved blocks and synced sections, if you store them yourself

Back up the affected tables before running the update. A short script that goes through all stored templates is usually enough. The examples below do the same replacement in JavaScript and SQL (table and column names are placeholders):

JavaScript

const OLD_PREFIX =
  "https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/";
const NEW_PREFIX = "https://s3.eu-central-1.amazonaws.com/my-company-assets/";

function migrateUrls(content) {
  return content.replaceAll(OLD_PREFIX, NEW_PREFIX);
}

// db.templates stands for your own data access layer
for (const template of await db.templates.findAll()) {
  await db.templates.update(template.id, {
    json: migrateUrls(template.json),
    html: migrateUrls(template.html),
  });
}

SQL

UPDATE templates
SET json = REPLACE(json,
             'https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/',
             'https://s3.eu-central-1.amazonaws.com/my-company-assets/'),
    html = REPLACE(html,
             'https://storage.googleapis.com/topol-io-plugin-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/',
             'https://s3.eu-central-1.amazonaws.com/my-company-assets/');

Template JSON stored as a parsed object rather than a string needs serializing first: JSON.parse(migrateUrls(JSON.stringify(json))).

Escaped slashes

PHP's json_encode escapes forward slashes by default, and other serializers can do the same. The stored JSON then contains https:\/\/storage.googleapis.com\/topol-io-plugin-..., and a plain search for the old prefix finds nothing. Check how the URLs look in your stored JSON and search for that exact form.

Leave shared editor assets unchanged

Some images in templates come from Topol's shared editor assets, such as the icons in Social blocks. Their URLs start with:

https://s3-eu-west-1.amazonaws.com/ecomail-assets/

These files aren't part of your storage and aren't in the export, so leave these URLs exactly as they are.

Step 5: Verify the migration

A search for the old prefix across everything you updated should return zero:

SELECT COUNT(*)
FROM templates
WHERE json LIKE '%topol-io-plugin-%'
   OR html LIKE '%topol-io-plugin-%';

Searching for topol-io-plugin- alone also matches the escaped form. Then open a few migrated templates in the editor and check that their images load and that your users' files and folders appear in the File Manager.

Emails that were already sent

Emails that were already sent can't be changed and keep loading images from the original URLs on Topol's storage. That's why we keep the original files there, so the images in those emails won't break. These emails may still generate a small amount of data traffic on Topol's storage, which decreases over time as older emails get opened less often.

If you hit any issues, contact our support.