Skip to main content

Alpha

Remember, this tool is an early alpha demo.  It is not feature complete and is intended to show off the capabilities.

Known Issues​

  • Generating large masked videos for export can cause your browser to timeout or freeze. We're working on a fix, but for now the workaround is to just export the mask and use a local tool to make a masked video.

  • The first mask always takes longer to generate and may take up to a minute. Later masks will be generated much faster, around 2-4 seconds per frame. We're working on improvements.

  • When you first load a video, it may take a minute or two before you can view other frames in that video. This is expected, as the backend takes a little while for the backend to process a video after load. If you try to view another frame and it just stays blank, please go back to the earlier frame and wait for a moment before viewing other frames.

  • There may be slowdowns related to usage, please let us know if you experience any excessive delays.

  • When making a lot of mask changes, scrubbing through lots of frames, or any other tasks that cause a lot of AI calls, it may develop a large backlog of tasks to process. This may cause slowdowns as the backend processes the queue. Please be patient in those circumstances.

Tutorial Video

Usage

To access the Alpha, click the first link that was included in your email. It will take you to your personal version of the demo which you can use.

Loading a video​

There are several ways to open your chosen video.  You can drag and drop the video into the video window or you can click the "Browse" or "Choose a file" button to select a video from your computer.  Once you select a video, it'll load the video and be ready for later steps.

To navigate a video, you can use the scrubber at the bottom, enter the exact frame that you want to go to in the text box on the left, or you can use the buttons to navigate through the video.

Making a rotoscope layer​

To make a Prompt layer you should add text to the prompt box based on what you want to have masked.  Provide a simple but clear description of what you want masked "A person" is better than "A skater wearing a red hat" but either can be used depending on the context.

To use points, instead create a new mask by entering a "Layer Name" and pressing "New Mask" (if you don't put in a name, the AI will create one of its own, but it'll be easiest to keep track of by naming it yourself).  Then click to add points you want to include in the mask, hold shift and click to add a negative point that will exclude that from the mask, or hold ctrl or alt to click to remove a point.

Exporting a video​

When you click either "Export Mask video" or "Export masked video" It'll process and export a video with all the masks in their own separate videos.  So for example, if you have 2 masks "Person" and "Dog" then you'll get back 2 mask videos, one containing the person and one containing the dog.  Clicking "Export masked video" will export both the mask video and a video where the masked object is cut out from the video and returns just the masked person.

Tips and Tricks

AI is a tool, not a panacea, it helps you but cannot do all the work of a professional.  To get the best results, you'll need to know how to make it work.

  • Exporting a video will only be output masks for frames that have already been processed. If you're missing any frames, just make sure to view them in the masker tool so that it generates the mask so it can be exported.

  • Try to use a prompt first, prompt is faster to propagate and generally easier to get exactly what you want

  • If you have something that can't work with a simple prompt, try to split up the task as much as possible.

  • Use a prompt where feasible, switch to point based when you can't use a prompt.

  • Go ahead in the video to find the end of where you want to modify and set a keyframe, then go back and delete the mask or modify it as needed.

  • To rename a mask or change a prompt, change the text in the box and then hit the button.

  • When using points, you'll sometimes get a grid-like effect on an area of the mask, this is where the AI is unsure if that should be included or not.  Generally it's best to make sure that the points it has aren't contradicting others, if they are, delete some bad points and add more points (either positive or negative) to make it more clear to the AI which objects should be masked.

  • Adding too many points makes it more likely you'll end up with the grid-effect.  In those situations check to remove points as soon as possible

  • Export will export one video for each layer that you have made anywhere in the video, but each frame doesn't have to have the same masks, where a mask is not in a frame it will return a blank mask for that frame.

  • Keyframes can be used to speed up the process.

  • Use a keyframe to stop a mask from propagating over a frame.  If you're masking something in a video that moves in and out of the video, you should probably mark the frame after the last frame as a keyframe before going back and setting the mask.

  • To fix one or a few frames, go ahead to where the problem stops and add a keyframe before going back to fix it.