In their creative process, designers employ techniques and strategies, moving from the abstract to the concrete through different physical and virtual means of representing form. Changes between the virtual and the physical are not always fluent. It would be a powerful tool for augmenting the design process if the designer was able to register and save each iteration of the physical and virtual models – a technique we call synchronization. We propose a novel method called Differential 3D Scanning that allows designers to save and update their models throughout the entire design process. The key idea is to use 3D scanning to automatically detect changes in the physical model, and to reflect them into the virtual model. During this process, the use of digital fabrication and 3D scanning devices could accumulate reconstruction errors which could be narrowed to the changed regions by our method. This technique helps to speculate about unfinished great works of art.
Hattab, Ammar, Ian Gonsher, Daniel Moreno, and Gabriel Taubin. "Differential 3d scanning." IEEE computer graphics and applications 3 (2017): 43-51.
As an example of our method, we did some experiments to complete the unfinished work of Michelangelo sculpture. At first we obtained a high quality scanned model of the statue from the digital Michelangelo project which has around 2 million vertices. Then we used a milling machine to carve this model in foam. Then a designer used sculpting tools to carve and finish the unfinished parts of the model physically. At last we used a 3D scanner to scan the modified model, and we applied our method to get the final virtual model.
In the case when the physical changes are smooth (As captured in the scanned point cloud). We propose a deformable model, where we deform the reference model until it matches the point cloud.
[Try it online]*note: click DeformMesh button to start the deformation.
We propose a new method for the rough alignment of point-clouds. We focus on a special type of point-clouds that is composed of simple geometric shapes like planes, cylinders, cones, etc. We call them 3D CAD point clouds. They are usually used in industrial and mechanical applications. The proposed method starts by detecting basic shapes in the point-clouds. And then using them to find the best transformation (rotation and translation) that aligns the point-clouds.
Hattab, A. and Taubin, G., 2018, March. 3D Rigid Registration of Cad Point-Clouds. In Computing Sciences and Engineering (ICCSE), 2018 International Conference on (pp. 1-6). IEEE.
We attached a ring of RGB LED lights to the 3D printer. To use it for the photometric stereo experiments; we turn on one LED and capture an image, and we repeat that for each LED.
Beside the direct photometric stereo, we are using the LEDs to detect sharp edges in the scanned 3D object in order to preserve them during the surface reconstruction and reverse engineering.
Example
Integrate 3D Laser Scanning with 3D Printer
We attached a line laser and a USB camera to the 3D printer. And we used the 3D printer mechanical movement for 3D scanning. This integration enables many applications. And the resulting point cloud is more uniform compared to turntable scanners.
To understand the 3D scanning process, and to find the best structure for the 3D scanner, we built a 3D laser scanning simulator; where the user can upload his own 3D model and then set the position and orientation for a number of laser lines, and the camera, then he can start the simulation. The result point cloud will try to approximate the real 3D scanning result.
For my PhD research, I started with the K8200 3D printer known as the most hackable open source 3D printer. we did several upgrades and modifications:
Upgraded the Z-Axis to solve the wobble problem.
Upgraded the extruder to use Titan extruder.
Upgraded the hotend to use E6D.
Attached Laser line and USB Camera.
Attached Lights ring for photometric stereo.
Other Projects in Brown University
Sketch Recognition Using Vector Graphics
, with Prof. James Hays, Brown University. (2015)
We explored three different ways to use vector graphics for sketch analysis and recognition, the first is to extract usual low level features like HOG but using vector graphics, then the use of global features, and at last the use of curve matching algorithms to match different sketches.
Using Local Moment Invariants for Partial 3D Shape Matching and Retrieval
, with Prof. Gabriel Taubin, Brown University. (2013)
Moment Invariants are normally used as Global shape descriptors, the goal of this project is to use them as a Local shape descriptor, and to build a 3D shape retrieval and matching system based on moment invariants.
USB Video Switch
, July 2013, with Prof. Gabriel Taubin, Brown University.
In this independent research project that lasted for two semesters I have
built a low cost USB video switch device, which can combine the video from
multiple UVC compliant USB cameras to emulate a real UVC webcam gadget
(capturing live video from different video capture source), I have
used BeagleBoard Xm and 4 Microsoft LifeCam 5000 USB cameras
Enhancing Bezier Curves Rendering
: May 2013: Performance of Bezier Curves Rendering in Web Browsers,
with Prof. Mark Ainsworth, Brown University.
I studied different algorithms of drawing Bezier curves, and
I proposed a new algorithm for enhancing the rendering
of Bezier curves by cashing the intermediate lines segments that approximate
the curve; by cashing them between subsequent animation steps and measured
the enhanced performance, and evaluated the result
I have implemented a special algorithm for body parts detection using
linking SIFT features between body parts and training data, I have then
evaluated the algorithm, and compared it with other body parts detection
algorithms.
I have implemented Image Inpainting algorithm where the user select the
parts to remove from an Image and the program will cover them from the
surrounding pixels.
Immersive Reality Control
: Improving the Control Behavior of Unmanned Ground Vehicle (UGV) using
Virtual Windows with Prof. Rosidah Sam, UITM University, Malaysia.
[Paper]
Accelerated Voice Calculator
I have accelerated a Command Recognition system using FPGA (Altera DE2
board), we have started with an open source C code
for voice recognition (that recognizes isolated words) then we ran that
code on NIOS II soft processor, then we accelerated that code by converting
parts of it to hardware. In the final demo we presented a Voice Controlled
Calculator that performs addition, subtraction, and multiplication on two
numbers, all controlled by voice.
POV Cylindrical Display
In this project which lasted for two semesters I was part of a team of
students, we tried to build a cylindrical display using a rotating arm of LEDs
and persistence of vision phenomenon to produce the display, and I was
responsible for software part in this project team, (verilog-FPGA, C on
the microprocessor, and C# on the PC).
Projects in Yarmouk University
3D Printer
from scratch!
I developed a 3-Axis 3D CNC carving machine, which can carve in flower
foam (or other Materials like wood) some 3D Model Drawn by the user (with
some limitations). I have also written software that will drive the machine.
the software allows the user to load standard 3D Model File (.X Format)
which contains the 3D Model; the software will then apply some algorithms
(Computer Aided Manufacturing Algorithms CAM) to the 3D Model to find a
complete 3D path for the carving head (called Toolpath), I have implemented
contour parallel and direction parallel CAM algorithms , the computer will
send this complete path in a standard form (like GCode) to a Microcontroller
connected to it (rabbit 3000), the microcontroller will send commands to
3 stepper motors (one for each axis) and to the head to follow the path
and carve the wanted model.
* I developed the machine from scratch (by combining old
printers to form the structure, connecting the driving circuits on bread
board (and trying different circuits), programming the microcontroller
to drive the circuit and connect to computer, and writing the software
on Visual C#
Hellium Balloons Vehicle
I have developed a Computer-Controlled Helium Balloons system, where the
user could control the movement of the Flying balloons wirelessly from
the computer, the supporting software allows the user to draw a 3D map
of the area where Helium balloons fly. It also allows the user to select
a target point, and the system will autonomously move Helium balloons to
that point, (My Project in its current state accepts only simple paths).
A wireless camera was attached to Helium Balloons, and a live view of the
camera is seen in the supporting software, The supporting software apply
some Image Processing Techniques on this live video and it identifies any
white line path on the ground.
VisualLife is an initiative to visualize everything about life using a network of interconnected visualizations, like Tree, Pie, Treemap, and Zoomable views.
VisualLife has data about a total of 1.8 million species, with about 40,000 images, the data is collected from several sources including "Catalogue of Life".
VisualLife is a built using a collaborative framework for interactive visualizations, so everyone is invited to contribute their visualizations about life.