python爬取网页,当爬到最后一页,没有下一页的情况下,出现list index out of range的错误原因及解决方法

原来代码是:

next_page_link = selector.xpath("//div[@class='nav-previous']/a/@href")[0]
print(next_page_link)
max_page_size = max_page_size + 1
if max_page_size > self.end_page:
    break
if next_page_link == False:
    active = False    //结束while循环

到了最后一页时,因为没有下一页,所以selector.xpath("//div[@class='nav-previous']/a/@href")得到一个空数组, 再在后面加上[0],则出错:索引超出范围。

修改到代码是:

next_page_link = selector.xpath("//div[@class='nav-previous']/a/@href")
print(next_page_link)
max_page_size = max_page_size + 1
if max_page_size > self.end_page:
    break
if next_page_link:
    next_page_link = next_page_link[0]
else:
    active = False   //结束while循环

测试通过