Extracting Data from Difficult Websites

https://www.youtube.com/watch?v=UiOSEhDT7Pc
Screen scrapers and data mining bots often encounter problems when extracting data from modern websites. Obstacles like AJAX discourage many bot writers from completing screen scraping projects. The good news is that you can overcome most challenges if you learn a few tricks. This session describes the (sometimes mind numbing) roadblocks that can come between you and your ability to apply a screen scraper to a website. You'll discover simple techniques for extracting data from websites that freely employ DHTML, AJAX, complex cookie management as well as other techniques. Additionally, you will also learn how "agencies" create large scale CAPTCHA solutions.

data, screen, websiteswebsite copying, copying website content, how to clone a website, website cloning, cloning a website, copying entire web content, downloading entire website,...2009 Hacker Dc17 Def Con Def Con Las Vegas Defcon Convention Conference Hackers Security, 2009