Lab 10: Processing Oblique UAS Imagery Using Image Annotation

Introduction

To date, we have been gathering imagery in nadir format. According to photogrammetry.com, nadir is defined as the camera pointing straight down. While this is a great means for producing orthomosaics and DSMs in bundle block software such as Pix4D, this method does not do well capturing vertical structures/objects. Therefore, the camera must be flown at different non nadir (aka oblique) angles in order to produce optimal results. Shown in Figure 1 is a UAV flying in nadir and a UAV flying non nadir. 

Figure 1: Nadir vs Non Nadir Flight

Since the geospatial world is moving rapidly towards being exclusively 3D being able to produce 3D datasets and incorporating the data sets  into a GIS is a strong market trend. In this activity, I am going to perform a quick demonstration of processing Oblique UAS imagery. To anyone that has don't this, I envy you. Unless you have a passion, patience, and a robust data processor, you will likely have to spend alot of time waiting for the data to process and potentially experience slow results. Nevertheless, processing oblique imagery can yield spectacular results when done with the right attitude software, data etc..,

Questions to Consider

In this section, it is important to cover other important topics that will help enrich the understanding of this lab. Below are the topics in question and answer format. 

What are some advantages and disadvantages of mapping nadir vs oblique angles? 

• Advantages of Nadir
• No need for varying camera angles . 
• Nadir missions can be laid in any orientation
• Mission pre-planning does not have to account for potential varying obstacles
• Disadvantages of Nadir
• It does not capture vertical surfaces
• Contains less information than oblique image collection
• Nadir imagery are less likely to process 3D models of vertical structures
• Advantages of Oblique 
• Ideal for 3D objects. 
• 3D models typically have more information than nadir models
• It can also still capture some horizontal surfaces
• Disadvantages of Oblique
• Not alot of autopilot options
• Depending on the subject, it could require beyond light of sight risk
• Unwanted data from oblique processing could occur which will therefore require image annotation

What is Image Annotation? 

According to Pix4D Support, Image Annotation is a feature that removes an object that appears in some images, the sky for oblique images, a constant object that appears in all images, or the background of the Orthoplane.

What Phase of Processing is Image Annotation Performed? 

typically, this is performed in the Initial Processing Phase. However if data is already available, one can annotate the images again after adding a point cloud and mesh. In this lab, I constantly had to reprocess data in order to see the differences. 

What types of Image Annotation are there? 

Referring to the on Pix4D help section  mask, carve and global mask are three main types of Image annotation. Below is a description of each, and examples of what they apply to:

• mask: this tool can be used to remove from the point cloud an obstacle that appears in few images, like a scaffold. All the images in which the object appear should be masked and step 2 should be rerun after masking.

• carve: each annotated pixel has an impact on the point cloud. The effect of the annotation is immediately visible in the 3D View of the rayCloud and it is easy to notice if some pixels were erroneously annotated so as to be corrected immediately.

• global mask: can be used to remove an obstacle that covers the same pixels in all images, like the foot of a drone. Only one image should be annotated and the annotation is propagated in the same pixels of all the images. If you have multiple objects to remove you can do it by annotating different images, one per object. However, in case that the objects appear in different locations on the images (different image coordinates), it is recommended not to use this tool as it will remove also other important information. In such cases, the mask tool is a better and safer option, even if it is more time-consuming

Methods

Like previous labs, I will use a tutorial style format to explain my findings while processing oblique imagery. In an effort to focus on the oblique imagery data processing, I have hyperlinked other labs to refer to in case you are interested in more specific information. 

1. Collect Oblique Imagery- knowing that you will need multiple flights, take into consideration the height of the structure with respect to the angle of your camera. In Figure 2 is an example of different camera angle options around a subject taken from. Drone Deploy.com

   

   Figure 2: Different Camera Angle Options

2. Log Into Pix4D Mapper and create a new project as shown in Figure 3. 

   

   Figure 3: Creating a New Project in Pix4D Mapper

3. Select and Upload your images- this had been demonstrated in lab 6. Therefore, feel free to reference the figures there in case you want to see a more comprehensive demonstration.
4. Shown in figure 4, is a provided data set from a UAS that flew circular pattern over a truck. This was processed before using image annotation.

   

   Figure 4: Curricular Pattern of Collecting Oblique Imagery

    
5. Once the initial processing is complete click (on) the pencil Icon on the right side of the screen. You then have the option to use any of the image annotations that I previously listed. 
6. After you decide your image annotation that you would like to do, you will have to left click and paint around the area you want to rid of. In order to see a difference, I had to process, and annotate for several hours in order to get the truck to be by itself. Although this is nowhere near perfect, click on the video below to see the difference compared to Figure 4. 

   

   Video of Truck Click here 

For a bonus, in the following video is  annotated light pole data flown by a UAV and processed after several hours.

Video of Light Pole Click Here

Conclusion

Since this is first time I've experienced image annotation, I am still exploring the data-sets, the tools, and how to make the data appear more cleaner. Although the images appear less noisy, there is plenty of work left to do. Nevertheless, processing data and editing it with the image annotation tool is another item that will factor into the interpretation and analysis of UAS data. Now, instead of having a huge image with terrain in the background, the object of interest is clearly defined enabling stakeholders to focus in on it.