Using ChatGPT 4 To Crawl Web Pages & Links To Extract Data
Recently ChatGPT4 allowed the crawling of links and website to extract information. This is a feature that many users had been waiting for. It now make the possibilities to extract data and aggregate data and content at an almost impossible scale, possible.
What is possible now is endless, and those with an ounce of creativity can create and build extraordinary applications.
TL;DR Version
Sign Up To ChatGPT 4
If you haven’t already upgraded, you’ll need a Chat GPT 4 account. At the time of writing you can upgrade for $20 per month.
Add The Prompt With Website Link
Add a simple prompt to ask the LLM (large language model) to crawl or view the website link you wish to extract data from.
For example, here is a prompt to ask ChatGPT4 to crawl a Wikipedia page, and extract entities based on Google NLP, and then list the entities.
The AI Output
Below you can see the output from our prompt.
You can then ask it further questions or give it further prompts.
Ready to Boost Your Shopify Store?
Increase revenue with video upsells and dominate search rankings with AI-powered SEO.
Related Articles
Using Role Based Prompting For Better AI Answers
Role prompting assigns a specific persona to an AI model. This technique instructs the AI to adopt a certain expertise, like a doctor or a historian. The AI then generates responses from that character’s perspective. This process tailors the output, making it more specific, contextual, and useful for the user. Okay so, role prompting. It’s […]
What Is Few-Shot Prompting: Making AI Actually Do What You Want
Few-shot prompting is a technique that guides large language models. It provides the model with several examples, or “shots,” directly within the prompt. These examples demonstrate a specific task and the desired output format. The model learns from these shots to generate a more accurate and contextually relevant response. You’ve probably heard the term “few-shot […]
Building Scalable AI Systems with Modular Prompting
Modular prompting is a technique that structures prompts into distinct segments or modules, each targeting a specific task or behavior. It improves consistency, reusability, and control in large language model outputs by isolating context, instructions, examples, or goals into separate blocks. If you’ve ever felt the frustration of getting wildly inconsistent results from a large […]