30 Comentários

1. Why didn't you just use proxies ?

2. Luke 😅 dude 😮 bro😂

3. Another video where the title question never gets answered. Brilliant.

4. IDK if the bot you program have some sort of rate limiting or like a delay of 1sec between each request!!

5. But if you went through manually, it would be fine. But because you can do it quickly, it’s banned.

6. When you think you are the smartest 😂

7. So you were banned by applying the skills that those jobs require? Shouldn't you be hired?

8. You logged in. That's your mistake. You can't log in and scrape.

9. I don't understand why this is illegal or why anyone would even care. What's wrong with collecting data efficiently?

10. i think you are just scraping too fast. or collecting too much data from one ip, likely both. its like how there is a limit of 70 connections per day on linked in you just need to stay within the amount of data they allow you

11. Proxies ?

12. They have anti scraping measures now too. I mean the site basically useless if you dont scrape it because the search is literally dogwater and i found it was the only way to actually filter the results to get actually relevant jobs

13. So…just state it isn't illegal (in state law)

    I think all these companies need to grow up and realize they are sending us paper catalogues with webpages. When we get the page we can do the fuck we want with it (privately)

14. "Publicly available"

15. What if we try to make a fast way to scrap manually data?

16. So they can collect our data anytime anywhere but we can't do the same?

17. Hi, Elon Musk just said data is being "aggressively scraped" how would twitter be measuring that?

18. Go through a public dataset manually
    LinkedIn: 😄
    Go through a public dataset with a bot
    LinkedIn: 😠

19. You need to rotate ip’s and user agents to reduce chances of being caught and flagged as a bot

20. Me my question is : how did you do this web scraping stuff? I mean just show where to find the place to learn. I will dedicate 24h straight of my life to learn. I will be very happy. As a data neird, am going crazy of all of the flashy stuff on internet but with no value. Help me.

21. A few years ago I scraped data that was in the public domain, from websites around the world. I never had a problem with accessing the web pages. The problem was that the webpages changed. You had to constantly rewrite the scraping code, or change inputs to scraping tools. It might have cost less and reduced a lot of stress. Just by hiring low cost labor to manually input the data.

22. I don't get it … Its no different from a real person sitting there and copy pasting things all day. Or they want you to do it manually so you can suffer …

23. I already knew that thats why never tried with LinkedIn.
    There are Github projects for that as well but doesn’t come with warranty.

24. Data viewed by the public on the internet via a privately owned corporate site does not necessarily equal public data.

25. Just use a proxy.

26. Because of that ToS, now i scraping data manually for my client, and it was pain in the arse. Lmao

27. Step one…build a web scraper that mimics human browsing behavior.

    Or use APIs and get your data that way.

28. This was actually a project idea that I had for quite some time, to see job distribution in different states/countries, cross relate to salary by company from GlassDoor and all that, while researching, I discovered that there is an informal LinkedIn API, so you don’t actually need to scrape all the data, quite helpful

    There are a bunch of articles on Medium about it too

29. I make a weather API. But now it give me an error like you have been blocked because we have registered an unusual ammount of traffic from your IP address.

    So I can't finish my project because of this. How can I solve this issue

30. Scrape so fast, the backend crashes

Comentários estão fechados.