'Xpath'에 해당되는 글 1건

  1. 2012.07.24 PHP5: Screen scraping with DOM and XPath
php2012. 7. 24. 23:50

PHP5: Screen scraping with DOM and XPath

This tutorial is continuation from previous yahoo screen-scraping using PHP4 tutorial.
We will try different method using DOM and XPath which only supported in PHP5.

First, a bit knowledge of XPath is required. More about XPATH can be read on:

http://www.zvon.org/xxl/XPathTutorial/General/examples.html

Also there's small concern that using XPATH is a bit slower than pure DOM Traversal. Read Speed: DOM traversal vs. XPath in PHP 5
But i personally also think that XPath is neat and easier.

Let's start. First we diagnose document structure using Mozilla Firebug.
Try a very easy case, which is to grab the title "Top Movies":

Copy XPath using Firebug and get this query:

/html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr/td/font/b

  1. Firefox automatically fix broken html structure, and it also add tbody tag. So, we need to remove this tag.
  2. Only grab first row of table. Change .../tr/td/font/b into .../tr[1]/td/font/b

Now we get our first XPath query:

/html/body/center/table[8]/tr/td[5]/table[4]/tr[1]/td/font

Next harder case is to grab contents.

XPath query from Firebug is:

/html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr[2]/td[2]/a/font/b

  1. Same problem here. Firefox automatically fix broken html structure, and it also add tbody tag. Remove tbody tag from XPath query.
  2. Grab all row of table. Change .../tr[2]/td[2]/a/font/b into .../tr/td[2]/a/font/b

Final XPath query for content is:

/html/body/center/table[8]/tr/td[5]/table[4]/tr/td[2]/a/font/b

Now final step is to put all two XPath queries into few lines of code, and we're done:

PLAIN TEXT
PHP:
  1. <?php
  2.     error_reporting(E_ERROR);// | E_WARNING | E_PARSE);
  3.     include ('Snoopy.class.php');
  4.    
  5.     $snooper = new Snoopy();
  6.     if ($snooper->fetch('http://movies.yahoo.com/mv/boxoffice/')) {
  7.         $dom = new DomDocument();
  8.         $dom->loadHTML($snooper->results);
  9.        
  10.         $x = new DomXPath($dom);
  11.  
  12.         //  /html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr/td/font/b
  13.         $nodes = $x->query('/html/body/center/table[8]/tr/td[5]/table[4]/tr[1]/td/font/b');
  14.         echo $nodes->item(0)->nodeValue, "<br/>\n"; //Top Movies
  15.  
  16.         //  /html/body/center/table[8]/tbody/tr/td[5]/table[4]/tbody/tr[2]/td[2]/a/font/b
  17.         $nodes = $x->query('/html/body/center/table[8]/tr/td[5]/table[4]/tr/td[2]/a/font/b');
  18.         foreach ($nodes as $node) {
  19.             echo $node->nodeValue, "<br/>\n";
  20.         }
  21.     }
  22. ?>

'php' 카테고리의 다른 글

사이트 긁어오기  (0) 2012.07.25
snoopy class를 이용한 youtube 이미지 저장 (php)  (0) 2012.07.25
PHP. 웹페이지 자동 로긴해서 긁어 오기와 HTML 파싱 라이브러리  (3) 2012.07.24
mysqli_stmt_bind_param , mysqli .. prepare, bind_param  (0) 2012.07.11
php 이미지 경로 지정할때 참고해라  (1) 2012.07.11
Posted by 다오나무

티스토리툴바