selectolax.lexbor module¶
LexborHTMLParser¶
-
class
selectolax.lexbor.
LexborHTMLParser
(html)¶ The lexbor HTML parser.
Use this class to parse raw HTML.
This parser mimics most of the stuff from
HTMLParser
but not inherits it directly.Parameters: - html : str (unicode) or bytes
-
any_css_matches
(self, tuple selectors)¶ Returns True if any of the specified CSS selectors matches a node.
-
body
¶ Returns document body.
-
clone
(self)¶ Clone the current tree.
-
css
(self, unicode query)¶ A CSS selector.
Matches pattern query against HTML tree. CSS selectors reference.
Parameters: - query : str
CSS selector (e.g. “div > :nth-child(2n+1):not(:has(a))”).
Returns: - selector : list of Node objects
-
css_first
(self, unicode query, default=None, strict=False)¶ Same as css but returns only the first match.
Parameters: - query : str
- default : bool, default None
Default value to return if there is no match.
- strict: bool, default True
Set to True if you want to check if there is strictly only one match in the document.
Returns: - selector : LexborNode object
-
css_matches
(self, unicode selector)¶
-
head
¶ Returns document head.
-
html
¶ Return HTML representation of the page.
-
raw_html
¶ raw_html: bytes
-
root
¶ Returns root node.
-
script_srcs_contain
(self, tuple queries)¶ Returns True if any of the script SRCs attributes contain on of the specified text.
Caches values on the first call to improve performance.
Parameters: - queries : tuple of str
-
scripts_contain
(self, unicode query)¶ Returns True if any of the script tags contain specified text.
Caches script tags on the first call to improve performance.
Parameters: - query : str
The query to check.
-
select
(self, query=None)¶ Select nodes give a CSS selector.
Works similarly to the the
css
method, but supports chained filtering and extra features.Parameters: - query : str or None
The CSS selector to use when searching for nodes.
Returns: - selector : The Selector class.
Remove specified tags from the node.
Parameters: - tags : list of str
List of tags to remove.
- recursive : bool, default True
Whenever to delete all its child nodes
Examples
>>> tree = LexborHTMLParser('<html><head></head><body><script></script><div>Hello world!</div></body></html>') >>> tags = ['head', 'style', 'script', 'xmp', 'iframe', 'noembed', 'noframes'] >>> tree.strip_tags(tags) >>> tree.html '<html><body><div>Hello world!</div></body></html>'
Returns a list of tags that match specified name.
Parameters: - name : str (e.g. div)
-
text
(self, bool deep=True, unicode separator=u'', bool strip=False)¶ Returns the text of the node including text of all its child nodes.
Parameters: - strip : bool, default False
If true, calls
str.strip()
on each text part to remove extra white spaces.- separator : str, default ‘’
The separator to use when joining text from different nodes.
- deep : bool, default True
If True, includes text from all child nodes.
Returns: - text : str
Unwraps specified tags from the HTML tree.
Works the same as the
unwrap
method, but applied to a list of tags.Parameters: - tags : list
List of tags to remove.
Examples
>>> tree = LexborHTMLParser("<div><a href="">Hello</a> <i>world</i>!</div>") >>> tree.body.unwrap_tags(['i','a']) >>> tree.body.html '<body><div>Hello world!</div></body>'
LexborNode¶
-
class
selectolax.lexbor.
LexborNode
¶ A class that represents HTML node (element).
-
any_css_matches
(self, tuple selectors)¶ Returns True if any of CSS selectors matches a node
-
attributes
¶ Get all attributes that belong to the current node.
The value of empty attributes is None.
Returns: - attributes : dictionary of all attributes.
Examples
>>> tree = LexborHTMLParser("<div data id='my_id'></div>") >>> node = tree.css_first('div') >>> node.attributes {'data': None, 'id': 'my_id'}
-
attrs
¶ A dict-like object that is similar to the
attributes
property, but operates directly on the Node data.Warning
Use
attributes
instead, if you don’t want to modify Node attributes.Returns: - attributes : Attributes mapping object.
Examples
>>> tree = LexborHTMLParser("<div id='a'></div>") >>> node = tree.css_first('div') >>> node.attrs <div attributes, 1 items> >>> node.attrs['id'] 'a' >>> node.attrs['foo'] = 'bar' >>> del node.attrs['id'] >>> node.attributes {'foo': 'bar'} >>> node.attrs['id'] = 'new_id' >>> node.html '<div foo="bar" id="new_id"></div>'
-
child
¶ Alias for the first_child property.
-
css
(self, unicode query)¶ Evaluate CSS selector against current node and its child nodes.
Matches pattern query against HTML tree. CSS selectors reference.
Parameters: - query : str
CSS selector (e.g. “div > :nth-child(2n+1):not(:has(a))”).
Returns: - selector : list of Node objects
-
css_first
(self, unicode query, default=None, bool strict=False)¶ Same as css but returns only the first match.
Parameters: - query : str
- default : bool, default None
Default value to return if there is no match.
- strict: bool, default True
Set to True if you want to check if there is strictly only one match in the document.
Returns: - selector : LexborNode object
-
css_matches
(self, unicode selector)¶ Returns True if CSS selector matches a node.
-
decompose
(self, bool recursive=True)¶ Remove the current node from the tree.
Parameters: - recursive : bool, default True
Whenever to delete all its child nodes
Examples
>>> tree = LexborHTMLParser(html) >>> for tag in tree.css('script'): >>> tag.decompose()
-
first_child
¶ Return the first child node.
-
html
¶ Return HTML representation of the current node including all its child nodes.
Returns: - text : str
-
id
¶ Get the id attribute of the node.
Returns None if id does not set.
Returns: - text : str
-
insert_after
(signatures, args, kwargs, defaults)¶ Insert a node after the current Node.
Parameters: - value : str, bytes or Node
The text or Node instance to insert after the Node. When a text string is passed, it’s treated as text. All HTML tags will be escaped. Convert and pass the
Node
object when you want to work with HTML. Does not clone theNode
object. All future changes to the passedNode
object will also be taken into account.
Examples
>>> tree = LexborHTMLParser('<div>Get <img src="" alt="Laptop"></div>') >>> img = tree.css_first('img') >>> img.insert_after(img.attributes.get('alt', '')) >>> tree.body.child.html '<div>Get <img src="" alt="Laptop">Laptop</div>'
>>> html_parser = LexborHTMLParser('<div>Get <span alt="Laptop"><img src="/jpg"> <div></div></span></div>') >>> html_parser2 = LexborHTMLParser('<div>Test</div>') >>> img_node = html_parser.css_first('img') >>> img_node.insert_after(html_parser2.body.child) <div>Get <span alt="Laptop"><img src="/jpg"><div>Test</div> <div></div></span></div>'
-
insert_before
(signatures, args, kwargs, defaults)¶ Insert a node before the current Node.
Parameters: - value : str, bytes or Node
The text or Node instance to insert before the Node. When a text string is passed, it’s treated as text. All HTML tags will be escaped. Convert and pass the
Node
object when you want to work with HTML. Does not clone theNode
object. All future changes to the passedNode
object will also be taken into account.
Examples
>>> tree = LexborHTMLParser('<div>Get <img src="" alt="Laptop"></div>') >>> img = tree.css_first('img') >>> img.insert_before(img.attributes.get('alt', '')) >>> tree.body.child.html '<div>Get Laptop<img src="" alt="Laptop"></div>'
>>> html_parser = LexborHTMLParser('<div>Get <span alt="Laptop"><img src="/jpg"> <div></div></span></div>') >>> html_parser2 = LexborHTMLParser('<div>Test</div>') >>> img_node = html_parser.css_first('img') >>> img_node.insert_before(html_parser2.body.child) <div>Get <span alt="Laptop"><div>Test</div><img src="/jpg"> <div></div></span></div>'
-
iter
(self, include_text=False)¶ Iterate over nodes on the current level.
Parameters: - include_text : bool
If True, includes text nodes as well.
Yields: - node
-
last_child
¶ Return last child node.
-
next
¶ Return next node.
-
parent
¶ Return the parent node.
-
parser
¶ parser: selectolax.lexbor.LexborHTMLParser
-
prev
¶ Return previous node.
-
raw_value
¶ Return the raw (unparsed, original) value of a node.
Currently, works on text nodes only.
Returns: - raw_value : bytes
Examples
>>> html_parser = LexborHTMLParser('<div><test></div>') >>> selector = html_parser.css_first('div') >>> selector.child.html '<test>' >>> selector.child.raw_value b'<test>'
-
remove
(self, bool recursive=True)¶ An alias for the decompose method.
-
replace_with
¶ Replace current Node with specified value.
Parameters: - value : str, bytes or Node
The text or Node instance to replace the Node with. When a text string is passed, it’s treated as text. All HTML tags will be escaped. Convert and pass the
Node
object when you want to work with HTML. Does not clone theNode
object. All future changes to the passedNode
object will also be taken into account.
Examples
>>> tree = LexborHTMLParser('<div>Get <img src="" alt="Laptop"></div>') >>> img = tree.css_first('img') >>> img.replace_with(img.attributes.get('alt', '')) >>> tree.body.child.html '<div>Get Laptop</div>'
>>> html_parser = LexborHTMLParser('<div>Get <span alt="Laptop"><img src="/jpg"> <div></div></span></div>') >>> html_parser2 = LexborHTMLParser('<div>Test</div>') >>> img_node = html_parser.css_first('img') >>> img_node.replace_with(html_parser2.body.child) '<div>Get <span alt="Laptop"><div>Test</div> <div></div></span></div>'
-
script_srcs_contain
(self, tuple queries)¶ Returns True if any of the script SRCs attributes contain on of the specified text.
Caches values on the first call to improve performance.
Parameters: - queries : tuple of str
-
scripts_contain
(self, unicode query)¶ Returns True if any of the script tags contain specified text.
Caches script tags on the first call to improve performance.
Parameters: - query : str
The query to check.
-
select
(self, query=None)¶ Select nodes given a CSS selector.
Works similarly to the the
css
method, but supports chained filtering and extra features.Parameters: - query : str or None
The CSS selector to use when searching for nodes.
Returns: - selector : The Selector class.
Remove specified tags from the HTML tree.
Parameters: - tags : list
List of tags to remove.
- recursive : bool, default True
Whenever to delete all its child nodes
Examples
>>> tree = LexborHTMLParser('<html><head></head><body><script></script><div>Hello world!</div></body></html>') >>> tags = ['head', 'style', 'script', 'xmp', 'iframe', 'noembed', 'noframes'] >>> tree.strip_tags(tags) >>> tree.html '<html><body><div>Hello world!</div></body></html>'
-
tag
¶ Return the name of the current tag (e.g. div, p, img).
Returns: - text : str
-
text
(self, bool deep=True, unicode separator=u'', bool strip=False)¶ Returns the text of the node including text of all its child nodes.
Parameters: - strip : bool, default False
If true, calls
str.strip()
on each text part to remove extra white spaces.- separator : str, default ‘’
The separator to use when joining text from different nodes.
- deep : bool, default True
If True, includes text from all child nodes.
Returns: - text : str
-
text_content
¶ Returns the text of the node if it is a text node.
Returns None for other nodes. Unlike the
text
method, does not include child nodes.Returns: - text : str or None.
-
text_lexbor
(self)¶ Returns the text of the node including text of all its child nodes.
Uses builtin method from lexbor.
-
traverse
(self, include_text=False)¶ Iterate over all child and next nodes starting from the current level.
Parameters: - include_text : bool
If True, includes text nodes as well.
Yields: - node
-
unwrap
(self)¶ Replace node with whatever is inside this node.
Examples
>>> tree = LexborHTMLParser("<div>Hello <i>world</i>!</div>") >>> tree.css_first('i').unwrap() >>> tree.html '<html><head></head><body><div>Hello world!</div></body></html>'
Unwraps specified tags from the HTML tree.
Works the same as the
unwrap
method, but applied to a list of tags.Parameters: - tags : list
List of tags to remove.
Examples
>>> tree = LexborHTMLParser("<div><a href="">Hello</a> <i>world</i>!</div>") >>> tree.body.unwrap_tags(['i','a']) >>> tree.body.html '<body><div>Hello world!</div></body>'
-