About Tensor Flow
-Machine learning library
-Open Source
-Powers hundreds of products at Google
This talk
-Apples vs Oranges
-Demo
-Hello World
Solving an "impossible" problem
-Can you write code to tell the difference between an apple and an orange?
a. By Color
def classify(pixels):
#Compare orange vs apple pixels
What can possibly go wrong?
Green apple, black and white picture etc
What next?
#Try detecting texture
#Write code to handle pears
New problem?
Start over again
We see a picture, the computer sees pixels
Problems
Vision, Voice Recognition etc
Learn from experience and examples instead of manual rules
Deep learning, supervised learning
Programming + Statistics
Example Label
1.jpg Apple
2.jpg Orange
Collect examples -> Find patterns -> Classifier
Demo : Tensor Flow Playground --> Explore
How do you classify them/differentiate between orange and apple?
Draw a line between them that clearly separates the two
#Machine learning finds the best parameters
What if the points are distributed differently?
Draw a circle around it
More layers and neurons = More complicated
Tensor flow uses neural networks
Input | Hidden layer | Output
Deep learning - What types if problem can we solve?
Input | Output
Pixels : A lion.jpg | "lion"
Audio : xyz.wmv | "see at her"
query, doc |
Translation | Translated sentence
Pixels : picture | Caption for picture(Microsoft has a product for it)
Growing use of deep learning at Google
Introduction to Tensor Flow
Tensor - Multidimensional array/matrix (n-dimensional)
Scalar,Vector,Matrix, Cube..
s=7, v={1,2}
Flow : Graph of operations (like +) to do with data
Hello World
import tensorflow as tf
a=tf.constant(2.0)
b=tf.constant(3.0)
c=tf.mul(a,b)
with tf.session() as sess:
print sess.run(c)
Higher level APIs
Classifier - Using scikit (TFLearn -> Scikit)
Motivation
Using many CPU and GPU to train model
Problem
Writing optimized, distributed code is hard. Can we make this easier?
Analogy : NumPy - Performant numeric code with a high level API
import numpy as np
a=numpy.array([1,2,3])
print np.dot(a,a)
Python Frontend | C++ Frontend
----------------------------------------
Tensor Flow Core Execution Engine
----------------------------------------
CPU | GPU | Android | iOS | ...
#Look out for Content Bundles
Vision, Voice ---> Solution is Tensor Flow
Business Applications
Finance(Risk Modelling), Medicine(Diabetes) etc
Firebase LegoWorkshop
goo.gl/rkqcrd
Google IO Keynote - Sundar Pichai
#Search - Supreme and improved with time
Connects People -> Places -> Things
#Images and Videos -- Customized video search, searches from your gallery,activities etc
#Translation -- Dynamic tranlation, Place your phone over English text and see it converted to Hindi
#Machine Learning and UI - Google Assisstant (a 2-way dialog with Google, help to get things done )
#Conversational services - Getting things done in context
Each conversation is different and billion users can use this assisstant in different contexts
#Getting the Google Assisstant home - Google Home
Presented By Mario from Chromecast team
Google Home - Hands free, with voice/remote voice, wifispeaker, device available by the end of this year
Music,Entertainment, Questions, Manage everyday tasks -- Ask Google + Customizable
Voice Recognition Technology used -- Building on from the Chromecast audio
MUSIC AND ENTERTAINMENT
Access podcasts,music,send music to google cast, can control other speakers Eg. "Play XYZ in the living room"
EVERYDAY TASKS
Alarm, to do tasks, connect to smart home seamlessly, control lights, controlling tasks beyond home etc
ASK GOOGLE
Organizing questions, retrieve travel itinerary, daily schedules, delivery etc. Answers complex search questions
google.com/home
Future scope - 3rd party developers interacting with the assisstant.
Photos - Computer Vision - 200M active users per month
Communication being the core usecase; Sharing life's moments
Erik Kay -- Introducing Allo and Duo
Allo - Smart Messaging Apple
#Expression
-Whisper Shout (Slider for size on smileys to express emotion levels)
-Adding motion to pictures
-Suggesting replies on pictures (Machine learning/Unique to you)
-Smart replies with pics
#Google Assisstant (Presented by Rebecca)
-Brings out Google Cards(eg restaurants, its reviews, reservations etc)
-Don't need to go out of the app(Open table)
-Understand context @Google - chatting with google/online friends
#Security
-Icognito chats
-End to end encryption
-Private notofications
-Expiring chats
ALLO - Fast, Smart, Secure ; Brings richness of Google to your chats
DUO - One to one video calling app
-Performs well on slow networks
-Even before we pick the call, we can see the video
-Team that build WebRTC
Allo and Duo - Avaialable this summer
Dave Bruke - Android N Release android.com/n
What's new in N?
#Performance - for game developers, game API
-Modern 3D Graphics
-VULKAN
-Android Runtime
* Added JIT compiler
* Reduction in compiled code size
* Makes install faster
#Security
*Enhancements in N
-File based encryption instead of block level
-Media framework hardening
-Seamless updates - latest version of system updates
* Services from Google
-Safe pricing
-Google Play Security Testing
-App security improvement program
-Safety Net
#Productivity
-Multitasking - Clear all button
-Quick switch (Double tap recents button, right most)
-Multi window
*Split screen(long press recent,double apps in the screen)
*Emoji(72 new emojis, skin color tones, unicode emojis, women)
-Notifications
*Direct reply feature
*Long tap notification to customize
Data Quality Release of N android.com/beta
Clay Bavor - VR Team
Cardboard in 2014 -> Mobile, Approachable for everyone
DayDream - High quality mobile VR experience
#Smartphones
-Sensors,Display etc
-VR mode in N - Performance, low latency, VR System UI
#Headset and Controllers
-Comfortable headset
-Motion,flexible controllers
-Available this fall
#Apps
-Games
-Google Play Movies
-Street View
David Singleton - Android Wear 2.0
Access to info,people,health, customizable view
-Smart Reply
-Handwriting
-Keyboard
-Watch face
-Messaging
-Fitness
*Phone at home, wear a watch : Still works out
Google Fit API -> Streaming music -> Activity Recognition
android.com/wear
Jason Titus - Developer Product Group
-Shift to mobile
-Progressive Web Apps
-Accelerated Mobile pages
Stephanne - Android Studio 2.2-way
-Speed
*10x faster build with speed
*3x faster test with emulator
*New layout design
*Test Recording
-Smarts *APK Analyzer
*Layout inspector
*Expanded code analysis
*IntelliJ 2016.1
-Platform Support *Enhanced Jack compiler/Java 8 support
*Enhanced C++ support
g.co/androidstudio
Firebase - Develop,Grow,Earn
-Free and Unlimited
-Analytics
-Built for Apps
-Cross network attribution
-Integrated across firebase
*Analytics
*Cloud Messaging
*Notifications
*Crash Reporting
*Remote Configuration
*Dynamic links
firebase.google.com
Ellie Powers - Android Instant Apps
-Click link, goes to app without install
-Install only if you want it on home screen
Tensor Flow - Embedding Machine Learning
Goolge Cloud Platform
-Cloud Machine Learning
-Computer Vision
-Speech,language and translate API
Tensor Processing Units - Customized internet hardware (Leverage using Google Cloud Platform)
Machine learning - Diabetic Retinopathy
Other booths
-Nest : Smart bulbs, smart lock etc - Camera, Smoke detector,Lock, Thermostat(Energy Saving) are the hardware required
-Spaces
-MDL
Firebase Analytics
Some challenges to build apps
-Capturing Data
-Spotting trends
-Using data across stack
Firebase - Integrated Platform
#Develop - Realtime database authentication, could messaging, storage, hosting
#Grow - Notifications, App Indexing, Dynamic links, Invites, Adwords
#Earn - AdMob
-Free and unlimited
-Automatic event reporting
-Seamlessly integrated
Firebase + App = Configuration
Growth engine for your business
firebase.corp.google.com
TENSOR FLOW - Data making decisions
Resources available on tensorflow.org
Anything humans can intuitively decide. Image classification, voice recognition etc
Building Billions on the Web, PM of Chrome team - Tal Oppenheir @taloppenheir
-Way we access the internet has changed
-Reach ALL the users
-New users are from around the world goo.gl/YwybkJ
-How do I reach them?
-We all started using internet on desktop only. But we are now moving towards mobile only
-Different connections, connectivity types, Connectivity quality, Connectivity cost, connectivity frequency
goo.gl/x1kp03 and goo.gl.qjmom
Development challenges
-Installs
-APK size
-Up to date
The mobile web - APP
-Accessible without install
-Storage constraints
-Always up to date
Progressive Web Apps
-Discoverable by all
-Fast and smooth
-Loads quickly, even on 2G
-Web offline
-Progressively installed
-Easy to re-engage with
eg Flipkart - Great impact
Leveraging PWA for billions
#Cost
-Data server
-DPR,width,viewport width etc
-Saving data on ?
-modpagespeed.com // Page Speed Module
-3x data savings
#Connectivity
-Speed varies between request and response
-downlinkMax
-Service Worker (Client side proxy in JS)
-Works offline
#Complexity
-Time and Money
-Add to homescreen
-Notifications using deep learning
-Progressive Web Apps ensure UX is seamless even if you are offline. If you are offline, call to order mechanism after basic product information is shown offline.
TRY IT YOURSELF
Mini Mobile Device Lab - Download from github
-Understand cost(data) and time
-Simulating htrottling/ 2G etc using Developer tools(Ctrl J)
developers.google.com/billions
developers.google.com/web/billions
What's new in Android Wear 2.0?
-Tethered + Untethered
-Personal Expression + Utilty
-Quick replies + Conversations
-Activity Tracking + Insights
Developer Preview
1. New User Interface
-Various watch face on swipe left (Customizable with settings)
-Hardware button = back
-App launcher (Hardware button once in the watch face)
-Scroll up on a watchface for notifications
MDL for apps
-There was swipe left and down in 1.0. There is only swipe down in 2.0
-Wearable action drawer and wearable navigation drawer(left <-> right)
-MDL - Choose your primary color
-Wear apps are "real" apps
-System UI and mobile
2.Standalone Apps
-Cloud and Data - Bypass phone app, talk directly to the cloud
-Standalone apps must manage authentication
-Notify users directly from your wearable app
-Apps are downloaded from PlayStore
1.0 - Download app from PlayStore on phone
2.0 - Download app from Playstore either through phone or device
Data <-> OS <-> Watch face
3. Watch face
Short text type, Icons, Ranged value
4. Messaging
-Message in the stream; Auto predict, all in one screen
-Content intent for launching to an app
-Expanded notifications for app-like experience
-Launcher for quick access
-Partner to watch face
-Inline accress, reply, conversations, content intent, faster launcher
5. Fitness
Real time data update notifications
-Suggestions on contextual conversations
-Recognizing activities like deadlifts, squats and pushups
-Automatic activity recognition
g.co/wearpreview
Material Improvements - Nick Butcher
Principles
-Material is a metaphor
-Bold,graphic,intentional
-Motion
plaid.app.io
github.com/nickbutcher/plaid
Material Surfaces
-Tangible objects(layers, collapses over other screen, state pinned = true) - Hierarchy - Interaction cues
-Elevation on tap
Begin delayed transitions
Set action visibility
Set activated state
Patterns - Navigational transitions(nested scroll)
-Plaid translucent(status window)
-Window transitions
-Bold graphic design (Typography, Color, Style etc)
-Pallette color library
-Grids
-Layout - Metrics and keylines
-Style - typography - Different presentation
-Leaving tabbed spaces to prevent content cutting
-Material Meaningful Motion
Raises on tab, background expands, image moves and grows, content enters, animations with buttons
25 Principles of Mobile App Design
1. App Navigation and Exploration
2. In-app search
3. Commerce and conversions
4. Registration
5. Form entry
6. Usability and Comprehension
1. App Navigation and Exploration
Find nearby hotels, May want other options -> So give both options #Google Places API
-When app to web transitions, make them frictionless #Chrome Custom Tools
2. In-app search
-Prominently display search field
-Search icon instead of search text (User capture)
-Search Pattern - Material Design
-Creating a search interface
-Provide filter and sort options (Clarity)
-Adding custom suggestions
-Adding recent query suggestions
3. Commerce and conversions
-Enabling comparison shopping features
-Compare feature
-Making it easy to add and manage payment method
#Android Pay developers.google.com/android-payment
4. Registration
-Differentiate between sign-in and signup (Instead say sign in and register)
#Google Identity Platform
#Sign in for Android Pay bit.ly/android-payment
-Make password authentication a frictionless experience
#Fingerprint authentication bit.ly/fingerprint-authentication
#Smartlock passwords
#Sign in hints
5. Form entry
Build user friendly forms (with help text/title)
eg Moving form up as user fills the form
#Textfields MDL
#Places API - Place autocomplete
Match keyboard with required inputs(Numeric keypad for DOB)
#Specifying input method type
6. Usability and Comprehension
-Provide text labels and visual keys to clarify
-Visual information (show little labels)
#Text for icons and buttons
#Making applications accessible
-Ask for permission in context
eg Allowing permission in context etc. Give find places(Ask in context)
#Requesting permission at Runtime
bit.ly/BetterMobileUserExperience
thinkwithgoogle.com
Technology in US Elections
-Federal Commission Dataset
-Reddit comments
-Wiki page reviews
-GDELT(Global Data) Knowledge Graph
#Google Big Query
redash.datafondue.com/queries
bigquery.cloud.google.com/table
#Tableau(Built in support for big query)
Candidates in the media
Google Cloud Dataflow
Big Query - User defined functions
curiousgnu.com
pantheon.corp.google.com/dataflow
firebase.corp.google.com
