takePhoto<\/code> action for it:<\/p>\n@IBAction func takePhoto(_ sender: UIButton) {\r\n let picker = UIImagePickerController()\r\n picker.delegate = self\r\n let alert = UIAlertController(title: nil, message: nil, preferredStyle: .actionSheet)\r\n if UIImagePickerController.isSourceTypeAvailable(.camera) {\r\n alert.addAction(UIAlertAction(title: \"Camera\", style: .default, handler: {action in\r\n picker.sourceType = .camera\r\n self.present(picker, animated: true, completion: nil)\r\n }))\r\n }\r\n alert.addAction(UIAlertAction(title: \"Photo Library\", style: .default, handler: { action in\r\n picker.sourceType = .photoLibrary\r\n \/\/ on iPad we are required to present this as a popover\r\n if UIDevice.current.userInterfaceIdiom == .pad {\r\n picker.modalPresentationStyle = .popover\r\n picker.popoverPresentationController?.sourceView = self.view\r\n picker.popoverPresentationController?.sourceRect = self.takePhotoButton.frame\r\n }\r\n self.present(picker, animated: true, completion: nil)\r\n }))\r\n alert.addAction(UIAlertAction(title: \"Cancel\", style: .cancel, handler: nil))\r\n \/\/ on iPad this is a popover\r\n alert.popoverPresentationController?.sourceView = self.view\r\n alert.popoverPresentationController?.sourceRect = takePhotoButton.frame\r\n self.present(alert, animated: true, completion: nil)\r\n}<\/pre>\nHere we used an UIImagePickerController<\/code> to get an image so we have to make our ViewController<\/code> implement the UIImagePickerControllerDelegate<\/code> and UINavigationControllerDelegate<\/code> protocols:<\/p>\nclass ViewController: UIViewController, UIImagePickerControllerDelegate, UINavigationControllerDelegate {<\/pre>\nWe also need an outlet for the button:<\/p>\n
@IBOutlet weak var takePhotoButton: UIButton!<\/pre>\nAnd an image<\/code> var:<\/p>\nvar image: UIImage!<\/pre>\nWe also need to add the following in the Info.plist to be able to access the camera and the photo library:<\/p>\n
\nPrivacy - Camera Usage Description<\/code>: Access to the camera is needed in order to be able to take a photo to be analyzed by the app<\/em><\/li>\nPrivacy - Photo Library Usage Description<\/code>: Access to the photo library is needed in order to be able to choose a photo to be analyzed by the app<\/em><\/li>\n<\/ul>\nAfter the users chooses an image we will use another view controller to show it and to let the user start the processing or go back to the first screen. Add a new View Controller<\/strong> in the Main.storyboard<\/code>. In it, add an Image View<\/strong> with an Aspect Fit<\/code> Content Mode<\/strong> and two buttons<\/strong> like in the image below (don\u2019t forget to use the necessary constraints):<\/p>\n![\"\"](\"https:\/\/intelligentbee.com\/blog\/wp-content\/uploads\/2017\/08\/Screen-Shot-2017-08-05-at-13.34.59-1024x598.png\")
<\/p>\n
Now, create a new UIViewController<\/strong> class named ImageViewControler.swift<\/code> and set it to be the class of the new View Controller you just added in the Main.storyboard<\/code>:<\/p>\nimport UIKit\r\n\r\nclass ImageViewController: UIViewController {\r\n\r\n override func viewDidLoad() {\r\n super.viewDidLoad()\r\n\r\n \/\/ Do any additional setup after loading the view.\r\n }\r\n}<\/pre>\nStill in the Main.storyboard<\/code>, create a Present Modally<\/code> kind segue between the two view controllers with the showImageSegue<\/code> identifier:<\/p>\n![\"\"](\"https:\/\/intelligentbee.com\/blog\/wp-content\/uploads\/2017\/08\/Screen-Shot-2017-08-05-at-14.45.55-1024x598.png\")
<\/p>\n
Also add an outlet for the Image View<\/strong> and a new property to hold the image from the user:<\/p>\n@IBOutlet weak var imageView: UIImageView!\r\n\r\nvar image: UIImage!<\/pre>\nNow, back to our initial ViewController<\/code> class, we need to present the new ImageViewController<\/code> and set the selected image:<\/p>\nfunc imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {\r\n dismiss(animated: true, completion: nil)\r\n image = info[UIImagePickerControllerOriginalImage] as! UIImage\r\n performSegue(withIdentifier: \"showImageSegue\", sender: self)\r\n}\r\n\r\noverride func prepare(for segue: UIStoryboardSegue, sender: Any?) {\r\n if segue.identifier == \"showImageSegue\" {\r\n if let imageViewController = segue.destination as? ImageViewController {\r\n imageViewController.image = self.image\r\n }\r\n }\r\n}<\/pre>\nWe also need an exit method to be called when we press the Close button from the Image View Controller:<\/p>\n
@IBAction func exit(unwindSegue: UIStoryboardSegue) {\r\n image = nil\r\n}<\/pre>\nTo make this work, head back to the Main.storyboard<\/code> and Ctrl+drag from the Close<\/strong> button to the exit icon of the Image View Controller and select the exit method from the popup.<\/p>\nTo actually show the selected image to the user we have to set it to the imageView<\/code>:<\/p>\noverride func viewDidLoad() {\r\n super.viewDidLoad()\r\n\r\n \/\/ Do any additional setup after loading the view.\r\n imageView.image = image\r\n}<\/pre>\nIf you run the app now you should be able to select a photo either from the camera or from the library and it will be presented to you in the second view controller with the Close<\/strong> and Process!<\/strong> buttons below it.<\/p>\n![\"\"](\"https:\/\/intelligentbee.com\/blog\/wp-content\/uploads\/2017\/08\/Screen-Shot-2017-08-05-at-15.18.05-601x1024.png\")
<\/p>\n
<\/span>Detect Face\u00a0Features<\/span><\/h2>\nIt\u2019s time to get to the fun part, detect the faces and faces features in the image.<\/p>\n
Create a new process<\/code> action for the Process!<\/strong> button with the following content:<\/p>\n@IBAction func process(_ sender: UIButton) {\r\n var orientation:Int32 = 0\r\n\r\n \/\/ detect image orientation, we need it to be accurate for the face detection to work\r\n switch image.imageOrientation {\r\n case .up:\r\n orientation = 1\r\n case .right:\r\n orientation = 6\r\n case .down:\r\n orientation = 3\r\n case .left:\r\n orientation = 8\r\n default:\r\n orientation = 1\r\n }\r\n\r\n \/\/ vision\r\n let faceLandmarksRequest = VNDetectFaceLandmarksRequest(completionHandler: self.handleFaceFeatures)\r\n let requestHandler = VNImageRequestHandler(cgImage: image.cgImage!, orientation: orientation ,options: [:])\r\n do {\r\n try requestHandler.perform([faceLandmarksRequest])\r\n } catch {\r\n print(error)\r\n }\r\n}<\/pre>\nAfter translating the image orientation from UIImageOrientationx values to kCGImagePropertyOrientation values (not sure why Apple didn\u2019t make them the same), the code will start the detection process from the Vision framework. Don\u2019t forget to import Vision<\/code> to have access to it\u2019s API.<\/p>\nWe\u2019ll add now the method that will be called when the Vision\u2019s processing is done:<\/p>\n
func handleFaceFeatures(request: VNRequest, errror: Error?) {\r\n guard let observations = request.results as? [VNFaceObservation] else {\r\n fatalError(\"unexpected result type!\")\r\n }\r\n\r\n for face in observations {\r\n addFaceLandmarksToImage(face)\r\n }\r\n}<\/pre>\nThis also calls yet another method that does the actual drawing on the image based on the data received from the detect face landmarks request:<\/p>\n
func addFaceLandmarksToImage(_ face: VNFaceObservation) {\r\n UIGraphicsBeginImageContextWithOptions(image.size, true, 0.0)\r\n let context = UIGraphicsGetCurrentContext()\r\n\r\n \/\/ draw the image\r\n image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))\r\n\r\n context?.translateBy(x: 0, y: image.size.height)\r\n context?.scaleBy(x: 1.0, y: -1.0)\r\n\r\n \/\/ draw the face rect\r\n let w = face.boundingBox.size.width * image.size.width\r\n let h = face.boundingBox.size.height * image.size.height\r\n let x = face.boundingBox.origin.x * image.size.width\r\n let y = face.boundingBox.origin.y * image.size.height\r\n let faceRect = CGRect(x: x, y: y, width: w, height: h)\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.red.cgColor)\r\n context?.setLineWidth(8.0)\r\n context?.addRect(faceRect)\r\n context?.drawPath(using: .stroke)\r\n context?.restoreGState()\r\n\r\n \/\/ face contour\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.faceContour {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ outer lips\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.outerLips {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ inner lips\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.innerLips {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ left eye\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.leftEye {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ right eye\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.rightEye {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ left pupil\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.leftPupil {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ right pupil\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.rightPupil {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ left eyebrow\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.leftEyebrow {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ right eyebrow\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.rightEyebrow {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ nose\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.nose {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.closePath()\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ nose crest\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.noseCrest {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ median line\r\n context?.saveGState()\r\n context?.setStrokeColor(UIColor.yellow.cgColor)\r\n if let landmark = face.landmarks?.medianLine {\r\n for i in 0...landmark.pointCount - 1 { \/\/ last point is 0,0\r\n let point = landmark.point(at: i)\r\n if i == 0 {\r\n context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n } else {\r\n context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))\r\n }\r\n }\r\n }\r\n context?.setLineWidth(8.0)\r\n context?.drawPath(using: .stroke)\r\n context?.saveGState()\r\n\r\n \/\/ get the final image\r\n let finalImage = UIGraphicsGetImageFromCurrentImageContext()\r\n\r\n \/\/ end drawing context\r\n UIGraphicsEndImageContext()\r\n\r\n imageView.image = finalImage\r\n}<\/pre>\nAs you can see we have quite a lot of features that Vision is able to identify: the face contour, the mouth (both inner and outer lips), the eyes together with the pupils and eyebrows, the nose and the nose crest and, finally, the median line of the faces.<\/p>\n
You can now run the app and take some unusual selfies of yourself. Here\u2019s mine:<\/p>\n
![\"\"](\"https:\/\/intelligentbee.com\/blog\/wp-content\/uploads\/2017\/08\/IMG_3029-576x1024.png\")
<\/p>\n
I hope you enjoyed this, please let me know in the comments how did it go and if there are things that can be improved. Also, some pictures taken with the app wouldn\u2019t hurt at all\u00a0\ud83d\ude42<\/p>\n
You can get the code from here: https:\/\/github.com\/intelligentbee\/FaceVision<\/a><\/p>\nThanks!<\/p>\n","protected":false},"excerpt":{"rendered":"
Great stuff is coming from Apple this autumn! Among a lot of new APIs there is the Vision Framework which […]<\/p>\n","protected":false},"author":3,"featured_media":2833,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[75,79,80],"tags":[134,158,248],"yst_prominent_words":[1124,1133,1132,1131,1130,1129,1128,1127,1126,1125,273,1123,1122,1121,1120,1119,1118,1117,390],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/posts\/2823"}],"collection":[{"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/comments?post=2823"}],"version-history":[{"count":4,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/posts\/2823\/revisions"}],"predecessor-version":[{"id":133071,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/posts\/2823\/revisions\/133071"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/media\/2833"}],"wp:attachment":[{"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/media?parent=2823"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/categories?post=2823"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/tags?post=2823"},{"taxonomy":"yst_prominent_words","embeddable":true,"href":"https:\/\/intelligentbee.com\/blog\/wp-json\/wp\/v2\/yst_prominent_words?post=2823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}