- Published on
Not every site has an API - sometimes the data you need only exists as rendered HTML. Scraping a page in .NET with HttpClient and HtmlAgilityPack, XPath vs CSS selectors, and where a static HTTP request stops being enough.
Web Scraping
All Posts
- net (71)
- csharp (68)
- entity-framework (24)
- aspnetcore (20)
- efcore (14)
- aspnet-core (11)
- performance (9)
- authentication (7)
- identity (7)
- security (6)
- azure (6)
- deployment (6)
- sql-server (6)
- jwt (4)
- web-api (4)
- ai (4)
- react (4)
- design-patterns (4)
- minimal-api (4)
- angular (3)
- mediatr (3)
- repository-pattern (3)
- dotnet10 (3)
- database (3)
- bulk-operations (3)
- ci-cd (3)
- httpclient (3)
- configuration (2)
- serilog (2)
- logging (2)
- claude-code (2)
- llm (2)
- typescript (2)
- sql (2)
- dependency-injection (2)
- iis (2)
- docker (2)
- migrations (2)
- clean-architecture (2)
- background-jobs (2)
- azure-devops (2)
- pipelines (2)
- devops (2)
- ioc (1)
- route-guard (1)
- api-key (1)
- api-versioning (1)
- rest (1)
- auditing (1)
- app-configuration (1)
- feature-flags (1)
- application-insights (1)
- observability (1)
- azure-functions (1)
- serverless (1)
- cron (1)
- omniroute (1)
- ai-gateway (1)
- sqlite (1)
- crud (1)
- dapper (1)
- dotnet (1)
- decorator (1)
- app-service (1)
- hosting (1)
- windows-server (1)
- kubernetes (1)
- docker-compose (1)
- aspire (1)
- cloud-native (1)
- structured-logging (1)
- signalr (1)
- real-time (1)
- scripting (1)
- dotnet-10 (1)
- scaffold (1)
- inheritance (1)
- github-actions (1)
- concurrency (1)
- relationships (1)
- fluent-api (1)
- soft-delete (1)
- middleware (1)
- error-handling (1)
- polly (1)
- resilience (1)
- transient-faults (1)
- builder-pattern (1)
- fluentvalidation (1)
- pdf (1)
- questpdf (1)
- reporting (1)
- oauth (1)
- google (1)
- hangfire (1)
- password-hashing (1)
- cryptography (1)
- email (1)
- smtp (1)
- cors (1)
- validation (1)
- password-policy (1)
- mcp (1)
- claude (1)
- caching (1)
- imemorycache (1)
- minimal-apis (1)
- testing (1)
- unit-testing (1)
- xunit (1)
- moq (1)
- multi-tenancy (1)
- code-quality (1)
- static-analysis (1)
- ndepend (1)
- technical-debt (1)
- options-pattern (1)
- pagination (1)
- language-features (1)
- code-organization (1)
- source-generators (1)
- performance-testing (1)
- k6 (1)
- grafana (1)
- load-testing (1)
- quartz (1)
- worker-service (1)
- rag (1)
- embeddings (1)
- vector-database (1)
- csv (1)
- csvhelper (1)
- file-io (1)
- refit (1)
- api (1)
- authorization (1)
- ollama (1)
- local-ai (1)
- specification-pattern (1)
- migration (1)
- upgrade (1)
web-scraping (1)
- htmlagilitypack (1)