[lecture: my note] 소프트웨어 테스팅 클래스 (translation: Software Testing Class) at Fast Campus

Subject	Software Testing Class (link)
Place	Fast Campus Main Building 4th Fl.
Time	10:00 ~ 17:00 on August 25, 2015
Speaker	최준현 (Lindberg Choi)
Host	N/A (paid lecture: 150,000 KRW)
Focus	QA basics, testing secret

1. Lesson
1. origin of software bug
1. origin: vacuum tube
2. term definition based on IEEE610.12-90
1. Mistake: A human action that produces an incorrect result.
2. Error: A difference...between a computed result and the correct result
3. Fault/Defect/Bug: An incorrect step, process, or data definition in a computer program
4. Failure: The [incorrect] result of a fault
3. reasons for defect:
1. engineers not complying with spec requirements
2. product manager making mistakes in writing spec requirements
3. engineers developing on their own due to absence of spec requirements
4. engineers’ contrived interpretation in architecturing
5. ...
4. Apple Appstore’s Common App Rejections => App Store Review Guidelines
5. key principle:
1. defect + regulatory violation => disabled service
2. what is test process
1. process: method to effectively increase task productivity
2. software testing process: spec requirement analysis -> plan test plan -> create test case -> conduct test -> report result
3. relationship between planning and testing
1. deliverables of “planner”
1. SRS (software requirement specification), IA (information architecture), business process, wireframe, SB (story board)
2. quality management method for “planner” and product manager
1. testing internal users
2. reviewing proposal and development doc
3. writing structured doc
4. creating test case in the perspective of “planner”
3. black-box testing method
1. definition: Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. <source: Wikipedia>
2. example
1. equivalence partitioning
2. BVA (boundary value analysis)
3. state transition testing
4. decision table testing
5. use case testing
6. pairwise testing
4. bug finding type per testing step
1. understanding test cycle
1. standard cycle: BAT -> main test -> ad-hoc test -> confirmation test -> regression test
2. multiple test cycle: 1st (BAT -> main test -> ad-hoc test) -> 2nd (confirmation test -> main test -> ad-hoc test) -> 3rd (confirmation test -> main test -> ad-hoc test -> confirmation test -> regression test)
2. bug finding type per testing step
1. BAT (build/binary acceptance test): e.g. label differs per organizations: e.g. BAT at Samsung Electronics, dev to QA; dev review; hurdle at Nexon
2. main test: conduct TC, finding function related bug
3. ad-hoc test: exception test, negative action test (c.f. random: range is pre-determined; ad-hoc: testing under given time; monkey test)
4. confirmation test: confirming bug fix, finding new bug due to unblocked functions
5. regression test: testing previous feature due to bug fix; typically, new fix brings more bugs on previous feature
5. bug finding principle
1. principle of defect clustering
1. principles of testing <source: link>
1. Testing shows presence of defects
2. Exhaustive testing is impossible
3. Early testing
4. Defect clustering
5. Pesticide paradox
6. Testing is context depending
7. Absence-of-error fallacy
2. most bug frequent area: upgraded/updated feature (e.g. At Google, engineers check file fix frequency in unit testing)
2. magic of test data
1. example: multimedia data by size and file types, member information, actual database data (e.g. 1234567890; 12345678901234567890, 일이삼사오육칠팔구십; ~!@#$%^&^&*((), null, 123456789 0, foreign language text, negative test case), physical data (e.g. OTP, credit card)
3. targeting weak point
1. reason: e.g. lack of connection feature to other modules; proposal issue; lack of state update test; batch function issue; incorrect data issue from admin and back office? (c.f. translation issue; number inconsistency issue)
4. exception management
5. targeting per product quality level
1. many bugs at early development stage: focus on test case
2. multiple test cycles: focus on unblocked feature
3. stable quality: focus on UI/UX, compare similar feature from competitor, ask for testings to external user
6. mindset of developer and tester
6. tester secret

2. Personal Takeaway
1. types of bug: reliability, security, compatibility, usability, functionality, performance, language
2. read
1. “Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing” by Elizabeth Hendrickson
2. “역사속의 소프트웨어 오류” by 김종하
3. “지식 제로부터 배우는 소프트웨어 테스트” by 타카하시 쥬이치
4. “경쟁력 있는 소프트웨어 테스터가 되는 법” by 최준현
3. UML (unified modeling language): ?
4. app testing trends: e.g. testing with oscilloscope to check currents not to drain battery
5. magic of test data
6. secret: e.g. testing backdoor (e.g. CGV, Olleh, Auction sites for ticketing), leave the session
7. establishing QA process at startups by StyleShare (link)