[class: PM Camp@Fast Campus#16] 사용자 행동 데이터 분석하기 #4 (my note is WIP)

7. 1 수

사용자 행동 데이터 분석하기 #4

구글 애널리틱스에서 A/B 테스트하기 A/B 테스팅 개요 Google Analytics에서 A/B 테스팅하기

강규영

1. AB Testing [Gyuyoung Kang@Box and Whisker, Founder]
1. lesson:
1. in AB testing, need to determine KPI first
2. battery usage ⇔ attention of users
3. funnel : 3~5 % for funnel conversion is average, 10% is good
4. captology  (computer aided persuasion technology): trying to persuade humans by computers e.g. tunneling
5. hick’s law: item number ⇔ time spent for decision-making
6. forms: progressive engagement e.g. Amazon doesn’t make users type address when signing up
7. exploration exists for improving accuracy (c.f. exploration vs. exploitation)

2. personal takeaway
1. study hot topic author’s teacher’s book
2. read The Signal and the Noise: Why So Many Predictions Fail--but Some Don't by Nate Silver

[idea: contextual loading animation] let's help users stay within stream of thoughts!

idea: replacing the current typical and no-tasteful loading animations with web/app service's context-relevant and make-sense creative ones 

background: I bumped into many websites or apps where loading animation is typical circling ones (i.e. link).  That type of non-contextual animations interrupts and disconnects users' "flow" or stream-of-thoughts for a moment in finishing up what they wanted to accomplish by using the sites or apps.  Though creating a contextual loading animation may require some plannings but the effort will pay off.  For users at e-commerce sites, they can finish their shopping without the feeling of interruptions or distractions so that they can focus on purchasing the items that they had in mind.  For e-commerce businesses, since turnover per user, meaning users quickly buy items in their shopping carts instead of staying on payment page too long, can be slightly higher than their previous system, e-commerce sites' revenue can be increased. 

usecase: As soon as users select "Publish" icon (i.e. link) at Blogger.com, the site can display a printing process animation providing users an impression that their writings are being printed through a printing machine at a newspaper factory until users' writings are finished updating. 

[class: PM Camp@Fast Campus#15] 사용자 행동 데이터 분석하기 #3 (my note is WIP)

8주차

6. 27 토

사용자 행동 데이터 분석하기 #3

구글 애널리틱스의 고급 기능 활용하기 각종 사용자 정의 기능들 정규표현식으로 검색하기 세그먼트 만들기 구글 스프레드시트와 연동하기 분석 실습 #3

강규영

1. App Log Architecturing, Regular Expression, Google Sheet Linking [Gyuyoung Kang@Box and Whisker, Founder]
1. lesson:
1. Regular Expression
1. purpose: language finding string patterns
2. two things to note: 1:1 대응, 3 patterns
3. 3 patterns:
1. character class: \d (i.e. [0-9]), \w (alphanumeric: [A-Za-z0-9]), \s (space, tab)
1. e.g. 1\d, \w, \d\d\d\d/d\d/\d\d, \w\w\w\w\d\d
2. ? (0~1), + (1~), * (0~)
1. e.g. +, .+, 하기(\d )?.+::
3. ( | ), [ ]
1. e.g. (Elizabeth|Lizz), (아빠|형)[가이], (엄마|아빠)랑, [가나다abc123], [가나다a-e], [가-힣], /2015/0[1-5]/,  /2015/0[12345]/
4. e.g. email pattern: e.g. \w+@[^\s]+\.\w\w\w?
5. c.f. \\
2. bounce
1. bounce
2. skim
3. reader
3. customized report: link to my example
4. EDA (investigator role) -> CDA (judge role)
1. watch out: usefulness

2. personal takeaway:
1. c.f. Project Gutenberg: free texts (good for example text)
2. question: filter applies to all view?
3. prepare reproducible analysis via automation
4. read Exploratory Data Analysis (EDA) by John Tukey-> Confirmatory Data Analysis (CDA)

[class: PM Camp@Fast Campus#14] 사용자 행동 데이터 분석하기 #2 (my note is WIP)

6. 24 수

사용자 행동 데이터 분석하기 #2

구글 애널리틱스의 데이터 수집/가공 방식 이해하기 및 탐색적 분석 데이터 수집 이해하기 데이터 가공 방식 이해하기 각 지표의 계산 방법 이해하기 분석 실습 #2

강규영

1. Understanding GA’s data gathering and processing method and exploratory analysis [Gyuyoung Kang@Box and Whisker, Founder]

1. lesson:
1. key GA concepts:
1. log: e.g. time, user CID (client ID), document path (e.g. URL/screen name), type (screenview/pageview, event), IP address, location data (e.g. city)
2. metric and dimension
1. metric (blue color in GA): numbers containing unit info (e.g. % Exit , Bounce Rate, Pageviews; 1,300 person, 1,300 USD, 1,300 clicks)
1. pageview: to make PV useful, add dimension (e.g. content, time and location access) to metrics
2. active user: differs depending on company’s business decision (e.g. Beat’s count AU
2. dimension (e.g. green color in GA): “~별” e.g. Date, App ID
3. event related term: c.f. event category (folder-like: e.g. gesture), action (verb: e.g. eat), label (object: e.g. item), value (number: e.g. 10 apple, 100 apple)
4. Hypercube (or OLAP (online analytical process) (c.f. OLTP or online transaction process)) (c.f. OLAP cube: slicing, dicing, drill-up, drill-down)
2. game application data analysis
1. improving game metrics (ask which data is missing to make a decision-making and why do you want to know the data?):
1. where people die
2. which one killed them
3. when do they play
4. which device (screen size)/browser/language users use
5. any areas users being misled
6. 터치 vs. 키보드 조작의 난이도 -> 조작계 바꿔야 하는지 결정하기 위해
1. average-time-on page
2. player-eat-apple
7. 언어 설정별 행동 차이 -> i18l, l10n 할지 결정하기 위해
1. 첫 방문 세션에서의 도움말 페이지에서 exit rate
1. en
2. else
2. 첫 방문 세션에서의 첫 플레이에 걸린 시간
3. 첫 방문 세션에서의 첫 플레이에 먹은 사과 개수
8. e.g. 얼마나 빨라지면 사람들이 그만두는가? -> 너무 빠르게 어려워지면 사람들이 게임을 다시 하지 않으니까
9. 3분 이상 플레이를 한다 goal 달성률이 20% -> 난이도가 너무 높다 -> 난이도를 낮춰야 또는 골 수정하거나
10. 난이도가 어느 정도이어야 사람들이 재플레이를 많이 할까?
11. 3일 연속 접속자 없음
12. 튕기는 사람 by 디바이스 -> 디바이스 별 설명 추가

2. personal takeaway:
1. game should reflect real-life: e.g. game difficulty setting can follow real life distribution in wealth
2. watch out for positive feedback loop, redefine key metrics and provide negative feedback loop
3. reduce retire people, provide matching

[class: PM Camp@Fast Campus#13] 사용자 행동 데이터 분석하기 #1 (my note is WIP)

7주차

6. 20 토

사용자 행동 데이터 분석하기 #1

구글 애널리틱스 개요, 활용 사례 및 기능 기능들 둘러보기 Google Analytics 개요 및 활용 사례 기능 둘러보기 및 핵심 개념 소개 분석 실습 #1

강규영

1. Google Analytics basics, application, function [Gyuyoung Kang@Box and Whisker, Founder]
1. lesson:
1. wrong approach of focusing on pageviews needs to be avoided and why data analysis is important
1. purpose of below ad technique
1. “헉” or “설마”: hooking
2. moving ad: design of eyes follow moving object, error click
3. small x button: error click
2. reason: page view ⇔ revenue
3. loser: user, advertiser
4. winner: platform (e.g. Google ad network), publisher
5. solution: smart advertiser using conversion metrics
2. pitfalls of mechanical application of data analysis without intuitive judgment
1. benefit of intuition: in data analysis, intuition helps avoids local optimum and seasonal bias
2. how to obtain skilled intuition: set up mechanism and environment, requiring both regularity and quantitative feedback
3. pitfalls of A/B test: seasonal bias (e.g. users’ color preference of green and red in Halloween and Christmas season), staying within local optimum (due to hill climbing algorithm)
3. data analysis requires complex learning task
4. proper order of data analysis: action based quest -> questions -> data -> data analysis
5. user story writing method (avoid including functions): “As a <ROLE>, I want <FEATURE> so that <BENEFIT> e.g.  As a <bank customer>, I want <to withdraw cash from an ATM>, so that <I don't have to wait in line at the bank.>
6. log (or weblog): system status change history (URL) by time and user (& IP address)
1. web browser (or world wide web browser) is web client (or http client) (c.f. web or http server)
2. c.f. add event type on GA to track down activities (e.g. transaction) within a web page
7. GA:
1. how-to
1. for additional analysis, consider adding this
1. e.g. ga('require', 'displayfeatures'); ga('require', 'linkid', 'linkid.js')
2. menu
1. audience: 누가 (e.g. 어떤 사람들)
1. session: 방문회수 by given time
2. user: unique browser number not a user (e.g. Chrome, IE, app)
3. engagement metrics: e.g. new sessions, pages/sessions
1. number can’t give either positive or negative (c.f. in general, higher number may mean positive); thus, combine with other metrics)
2. acquisition: 어디에서 (e.g. push message, Facebook, etc.)
1. direct
2. referral: social (e.g. Facebook), search (organic, paid), email (e.g. Gmail), referral (others)
3. behavior: 뭘했나?
4. conversion: 얼마 벌었나? (e.g. purchase, download)
1. goal: selection of metric needs to influence people’s behavior
2. consider analyzing Top Conversion Path
3. try
1. modify bounce rate, filter IP for tracking
4. bounce rate: session base
8. read Extreme Programming Explained: Embrace Change by Kent Beck

2. personal takeaway:
1. new term: hill climbing algorithm, Return On Advertising Spend (ROAS)
2. 10,000 hour law:
1. to obtain skilled intuition, establish mechanism and environment, requiring both regularity and quantitative feedback
2. improve areas of weakness with focus
3. key concept of agile development methodology: deliver values to users everyday
4. try Google Data Saver for VPN
5. Chrome Extension: Table Booster
6. when communicating with software engineers, tell them what you want to accomplish (aka your intentions) instead of detailed function list 
7. consider implementing “careful reading” JavaScript into your website to track down accurate user feedback