Scraping Rightmove with a Python-base Desktop Tool
A couple of years ago I received an enquiry via my website from a local entrepreneur who specialises in properties. They utilise Rightmove very heavily in scouting and evaluating potential investments. As you could imagine, this workflow would require a great deal of manual work and data entry. Manually reviewing countless archives and pages through various data sources, including Rightmove, and then evaluating the various options later on.
They asked me whether I could build a tool that would handle as much of the data gathering as possible and I knew without a doubt that I could deliver just the tool they needed to take their operation to the next level.
Goals & Features
- Search Rightmove for active listings.
- Search Rightmove for historic listings.
- Get property sizes from an input CSV.
- Compile to a CSV file and save locally.
- Run on Windows 10/11 directly from a native executable (.exe) file.
Tech Stack
Demonstration
In the above video, you’ll see a full demo of the user interface of this tool. As the interface is built using Tkinter, Python’s most basic built-in GUI library, it doesn’t look modern. Luckily, it’s what’s underneath that counts here.
Rightmove active listings scan
The active listings scan will use the provided criteria to fetch a list of all results currently available on Rightmove. The results should match that which you’d get by manually traditionally searching Rightmove through their website.
It will then load up each listing and assemble some key information which is eventually saved in a CSV file of your choosing upon the scan finishing. Note: the information it returns isn’t an exclusive list of what’s available, but rather what was required by the client. This can be scaled to provide much more data if needed in the future.
Example CSV output with the query ‘FY7’.
Rightmove sales history scan
The sales history scan will use the provided criteria to fetch a list of historic sales belonging to a given postcode. Similarly to the active listings scan, this currently provides only the information required by the client at the time of development, but it could be easily scaled to provide much more data if required.
Example CSV output with the query ‘FY7’.
GOV
This scan type accepts a CSV file as an input, expecting at least two columns: ‘paon’ (house number) and ‘postcode’.
For each item in the input CSV, this goes to the UK Gov website’s Find Energy Certificate and, using the provided information, it finds the listing for that property if one exists, and pulls the size of the property from the energy certificate records (should it be listed).
While the active listings scan can sometimes get the property size, Rightmove doesn’t have that information for every listing. Using this approach, you can optionally re-process the outputs from previous scans to also get a decently accurate report on the property size.
Example CSV output with example 2’s CSV as the input.
Availability of this tool
Unfortunately, this is one of my projects, and while I can showcase it to you, I can’t provide the source code or a complete build in an open-source capacity.
With that in mind, if this sounds like something you need, don’t hesitate to reach out and contact me directly so we can collaborate on building the right tool for you.