Tìm việc dễ dàng...

Senior Software Engineer

Địa điểm

Hồ Chí Minh

  • Lương

    40 Tr - 50 Tr VND

  • Kinh nghiệm

    3 - 5 Năm

  • Cấp bậc

    Nhân viên

  • Hết hạn nộp


Phúc lợi

  • Chế độ bảo hiểm

Mô tả Công việc

Crawler Development and maintenance: Mentor junior developers in crawler development techniques and best practices, Provide guidance in architectural decisions for data crawling and management.

  • Technical Leadership: Analyze and improve the performance of crawlers, reducing latency and increasing data retrieval efficiency, Address challenges related to rate limiting and blocking mechanisms, Performance Optimization: Implement security measures to protect sensitive data and ensure compliance with data privacy laws, Monitor and adapt to changes in social media platforms’s APIs, rules, and regulations
  • Security and Compliance: Integrate crawlers with existing systems and third-party tools to streamline data access and usage., Work closely with other engineering teams to incorporate new features and maintain compatibility across services.
  • Integration and collaboration: Build data pipelines to clean, process and store data effectively, Collaborate with data scientists and analysts to ensure data quality and availability for analysis.
  • Data Management: Design, develop and maintain robust web crawlers that can effectively collect data from Facebook, Tiktok, Threads, Linkedin, and other platforms, Optimize crawling algorithms to ensure data is gathered quickly and accurately while adhering to ethical standards and platforms policies

Yêu Cầu Công Việc

Technical skills:

  • Strong programming skills in python, javascript, or the other relevant languages.
  • Hands-on experience with web scraping libraries and tools like beautiful soup, scrapy, selenium, or puppeteer.
  • Familiarity with API development and RESTful services.


  • Proven track record in building web crawlers for social media platforms
  • Understanding of platform-specific challenges (e.g., rate limiting, data structures,)

Data skills:

  • Experience working with big data tools (e.g., Hadoop, Spark).
  • Proficiency with SQL/NoSQL databases.

Thông tin khác

  • Độ tuổi: Không giới hạn tuổi
  • Lương: 40 Tr - 50 Tr VND
Lưu việc làm Gửi tôi việc làm tương tự Báo xấu

Báo xấu

Các công việc tương tự