3D scanning to 3D modelling to AR experiences
With regards to developing a production pathway that is compatible across 3D scanning, 3D modelling and augmented reality, we are in the early stages of testing and Hoverlay and Aero are both viable options.
Adobe Aero is a good choice because it allows for the animation of assets once you import them into the interface. There are also options for scaling, moving and rotating objects and assets and you can choose how the experience starts when the viewer opens the scene—autoplay, tapping the screen, proximity and through an image anchor. There is no limit on the size of assets but the scene will load much slower when large files are brought in. Some image mapping issues were also discovered with certain 3D object files coming directly from scanning apps and the modeling software we tested.
Aero was used with White Buffalo Youth Lodge from models scanned with WiDAR and sculpted and exported in SculptGL. Scanned models brought into SculptGL and manipulated or painted lost their original texture mapping and thus these manipulated scans could not be brought into Aero with their combined texture mapping (original scan and new paint material). Results in Aero can be seen on our community outreach page for White Buffalo Youth Lodge.
A great feature in Aero is that Adobe files (such as Photoshop and Illustrator) with layers can be brought in and these will be automatically displayed in 2.5D space. The app accepts photos, layered files, audio, or other 2D and 3D content in the following formats: PSD, JPEG, OBJ, GLB, FBX (on desktop), GIF, PNG Sequence, WAV, and MP3. Aero provides more options with formats of 3D assets than Hoverlay. The app claims to be fully free, but you do need to have an Adobe account to be able to create an AR experience. Viewing of the AR is activated through the scanning of a QR code that is generated and shareable when you build a scene. Aero works on both Android and iOS. Limitations are noted in situations where you would like to have multiple scenes or assets exist in the same exhibition as you need to go through the process of scanning the QR code for each individual work. Aero allows for image anchors so you can have the AR be triggered by a specific image but the app will not automatically switch to a new AR experience when you move the phone to another image trigger in the room while you are looking at a different experience. Each scene is independent. We found the scanning of Aero QR codes slightly more awkward than Hoverlay.
Hoverlay is a great option that we discovered a bit later into our process. Hoverlay allows one to geo-locate five AR “spaces” for free to a specific address/location (additional spaces over the five can be purchased with a subscription). Although Hoverlay doesn’t allow you to animate objects or assets in-app like Aero, it is very easy to use and also has a wide range of file formats that you can upload: PNG, JPG, MP4 (greenscreen video), MP4 (normal video), MP3, and GLB. Limitations are in the 3D file format of only being compatible with GLB (20MB limit) and in the 8MB file size limit for MP4. Some of the scanning apps and modeling software do not export in GLB (so it is important to check that the scanning app you are using will export in GLB if interested in Hoverlay). One feature that we like is that you can choose to allow your viewer to move, rotate and scale your objects in real time once they open your space. AR scenes can also be triggered via a QR code or anchored by an image trigger in the space. If you have the Hoverlay app on your phone you can find uploaded spaces in the current part of the world that you find yourself in, allowing artists to participate in a larger AR ecosystem and environment. You can choose not to geo-locate in which case your AR scene is only available to those that have a QR code (generated and shareable when you build a space).
At Mosquito First Nation, we used EZAudioCut(MT) Lite for recording and editing audio, and KIRI and WiDAR for scanning objects. MP3 files and GLB files were uploaded into Hoverlay to create spaces that held both 3D objects and audio (see the Outreach page for Mosquito Grizzly Bear’s Head Lean Man First Nation for more information).
It is also worth mentioning WiDAR. This versatile app offers a unique opportunity due to its multi-functionality, which includes 3D scanning of objects, 3D manipulation and composition options for scanned objects, and augmented reality experiences using scanned (and manipulated) objects. Therefore, WiDAR provides a unique production pathway in a single user-friendly, cost-effective app.
