Restoring Avid MediaFiles folders and Avid Projects from S3

This procedure creates automations to copy content from specific S3 directory to a specific existing media space.  

Note: that you only have to do this once.   After you create the automations, you can reuse them each time you want to restore something by changing the source media space folder on the media space node and by changing the destination media space in the copy node.

Automation 1- Restore Avid MediaFiles from S3 to an Existing Media Space

Create a new Flow Automation Template consisting of a Manual Trigger > Media Space Source > Copy Task.

Set the Media Space to the Flow Media Space for the S3 Bucket, and set the Folder to be AvidStyle/SourceMediaSpace_1 where "SourceMediaSpace_1" is the name of the media space in the S3 bucket that you want to restore with the _1 added at the end.  The exact name can be verified by browsing to the S3 Bucket media space in AirFlow.

Set the copy node Media Space to be the existing media space where you want the content restored.

Check the Preserve Path option.  No Folder is needed.

Note:  If you are making a new media space for this restore, DO NOT use the same name as any older deleted media space.  If you use the same name as the source media space add suffix of some sort to the end to make it unique.

To run the start the restore, simply run the automation by making sure it is selected in the template list, and then clicking the green play button.

Restore Avid Projects Automation

Only one automation is really needed, as you can re-use the one created above for the projects spaces by changing the source folder and copy destination. and running it again.

However, if you would like separate automations, you can duplicate the automation created above to create a separate one for restoring projects.   The only advantage to this is that you can run duplicates at the same time to set up and run multiple restores at once. 

To relocate the media do the following:

In Finder or File Explorer, open the destination media space and navigate to

"AvidStyle/SourceSpaceName_1/Content/Avid MediaFiles/MXF" and then note the machine folders present.

Select all of the machinename.1 folders and drag them to the active Avid MediaFiles folder at the root of the media space.  MediaSpace/Avid MediaFiles/MXF

Determine which current Avid machine you are going to use to re-index the restored media and check its share name in System Settings >Sharing on mac or System >About on Windows.

Change the names of all of the machine folders to the share name of that computer followed by a unique number. (don't include the .local on Mac machines)

In this example, I used a numeric string for the date to keep track of when I restored that media, but you can use any number as long as you do not end up with any folders that have the same machine name and number.

You are now ready to open Media Composer on that machine and let it index the new files.

Relocating the Avid Projects Folder

You can easily use the restored Avid Projects right where they are, but the depth of the restore directory may be inconvenient.   

There is one rule to remember when moving around Avid projects.  Do not rename the top-level folder or anything inside of it.  As long as you follow that rule, you can move project folders anywhere. In the example below, this would be the "Digital Prod Demo" folder inside the "Avid Projects" folder.

 If there is already an existing Avid Projects folder, you can use it as long as there are no projects with the same name.

Alternatively, you can rename and move the entire restored Avid Projects folder and keep the restored versions of projects separate.  

Find the Avid Projects folder in the preserved restore path.

Rename it to Avid Projects Restored

Move it wherever it is convenient for you to access.  In this example, I renamed it and moved it to the root of the media space.

Re-indexing the Avid MediaFiles machine folder is done automatically.  When an instance of Media Composer is launched, it scans all Avid MediaFiles folders that are mounted.   It re-indexes any folder that matches the machine that it is running on and updates the .pmr and .mdb (database files) in these folders making the media accessible to all instances of Media Composer that mount that drive.  

This also happen any time the user opens or accesses the operating system's file browser and then switches back to Media Composer.   (Finder Window for Mac OS and Explorer Window for Windows)

After this, indexing is complete, all master clips are automatically linked to any available .mxf files.

Note Avid Bin sharing must be enabled, so always click "yes" when presented with this dialog box.

To add master clips to new projects using the Media Tool first launch Avid Media Composer and open any project or create any new project.

Open the Media tool.

Select the media space in the drive column on the left and select All Projects in the project column on the right.  Also uncheck Linked Master Clips. Click OK.

You will see a temporary bin full of clips including all of the newly indexed clips, and the drive column will show the media space you selected.

If you switch to clip view in the Avid bin, you may see all the clips stacked up on top of each other.  Just click the hamburger menu, choose Align and Fill>Fill Window, and you will then see all the clips.

Treat the Meda Tool like any other bin.  Select clips, preview them, and drag the ones you want to use to a project bin in your project.

Access and copy master clips to a new project using Media Tool.

Open restored project from the location you copied it to.  You should see that all media relinks automatically.  No need to import or relink anything.

What are all those extra directories? 
The path contains all of the directories that are used to store the media on the RAID of your EditShare system, and you commonly do not see some of the directories when working with EditShare Connect.  However, when the files are backed up with Flex Sync, the entire path is preserved.

When "Preserve Path" is enabled, the automation copy node will copy the content selected in the source node and preserve the entire original path on the S3. 

The path for the restored files will begin with a fixed prefix

Example:  "AvidStyle/SourceSpaceName_1/Content/"

The restored content is found below the "Content" directory

Avid MediaFiles Folder Structure Explained

There may be many machine names inside the MXF folder.
In the Avid Media Composer shared media environment, each instance of Media Composer creates a folder in the MXF folder with its machine name followed by a . and a number. 

Examples: "mymachine.1"    "yourmachine.201"   "Tims_Macintosh.3"

Avid does this so that individual machines can use the managed Avid MediaFiles folder without overwriting each other's media and database files.  

Each instance of Media Composer is responsible for indexing the media files in the folders with its machine name.  

Once indexed, the .mdb and .pmr files each folder can be read by every other instance of Media Composer allowing them to locate the .mxf media files associated with any master clip and automatically link to it.   Each instance can also browse all media in all folders using the Avid Media Tool.  This allows any editor to access and use any media on any shared volume in any project.  

Avid Project Folders

General Project Folder Info

An Avid Project is actually a set of files organized in a project folder.  It contains the actual project file, bin files, logs, settings, search data, and waveform caches.  Any files that MC names with the same name as the project must remain that way to maintain the integrity of the project.

For this reason, it is not advised to rename the top-level folder for an avid project.   When restoring backups, it is possible to have two versions of a project with the same top-level folder name.  In this case, these must be kept in different places since two folders cannot exist in the same place with the same name.

Project Folders in a Shared Environment

When a project is on a shared volume where multiple instances of Media Composer can access it at the same time, corruption of these files can result unless a system is employed to keep different instances from writing to these files at the same time.

When AvidStyle is activated in EditShare Connect, it triggers Media Composer to implement Avids project sharing and bin locking features to prevent this corruption.   With project sharing enabled, each instance of MC writes settings and project files in its own folder. 

Since all users need to access the same bins, the sharing protocol locks bins giving write access to the first user to open them.  They get a green lock. 

Other users can only get read-only access when they open the bin.  They see a red lock.