این یک پروژه مهندسی داده پایان به انتها با استفاده از جریان هوا و پایتون است. در این پروژه، داده‌ها را با استفاده از API توییتر استخراج می‌کنیم، از پایتون برای تبدیل داده‌ها استفاده می‌کنیم، کد را در Airflow/EC2 مستقر می‌کنیم و نتیجه نهایی را در کد آمازون S3 و دستورات مورد استفاده در پروژه ذخیره می‌کنیم – منابع دیگر: Twitter API – Python (نصب و اصول) – راه اندازی حساب AWS – Airflow Doc – به داده ها با سرور Darshil Discord بپیوندید: Timestamps 0:00 معرفی پروژه 0:56 بخش 1 (پیش نیاز) 2:21 بخش 2 (درک مفهوم اولیه) 7:55 بخش 3 (اجرای پروژه) 👦🏻 لینکدین من – 📷 اینستاگرام – 🎯 توییتر – 🌟 لطفا برای مطالب شگفت انگیز بیشتر لایک کنید ❤️ و مشترک شوید! 🌟 3 کتابی که باید بخوانید 📈اصول: زندگی و کار: 👀 کار عمیق: 💼بازنگری: فناوری که من هر روز استفاده می کنم : 🔅 چراغ و پایه حلقه اوزاکا: 🎧هدفون سونی WH-1000XM4: 🖱صفحه کلید و ماوس Zebronics Zeb-War: 💺CELLBELL C104 صندلی اداری: 👉نقشه راه مهندسی کامل داده ها: 👉Data Serecties Projecties: Fulltime Data: : 👉Data With Darshil Podcast: ✨Tags ✨ پروژه مهندسی داده، پروژه کامل مهندسی داده برای مبتدیان، یادگیری مهندسی داده، نقشه راه مهندسی داده ✨ هشتگ ها ✨ #جریان هوا #مهندسی داده #پروژه

این مطلب توسط وبسایت آموزشی سوشال پورت گردآوری شده است.
منبع

لطفا ستاره بده (سمت چپ بیشترین امتیاز)
به اشتراک بگذارید:

46 نظرات

  • Abeg Patel, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thank you👍 for the awesome content We want more data engineering real world end to end project with real time streaming to deal with real world problem and situation.

  • Syed Mohammad Bilal, 1401-06-31 @ 11:08 ق.ظ پاسخ

    How to create access keys??

  • Mega Mind Gamerz, 1401-06-31 @ 11:08 ق.ظ پاسخ

    It's working thanks my friend

  • Raghavarao Sunkanapally, 1401-06-31 @ 11:08 ق.ظ پاسخ

    At first, I tried on AWS Ubuntu t2.micro to run airflow – Gave error(Didnot execute Airflow Standalone)
    But Later, I tried t3.medium, It worked well. Thanks for the video!

  • Mangesh Gawali, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Great explanation. Thanks, Darshil! 😊

  • santosh chavan, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Awesome sir…… That's what I actually want…

  • Mohammad Tanvir Hossain, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hey bro always liked your video !! do create more project based Data Engineering Videos. Would be nice if you would create something that will teach about docker.

  • sudha rani, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Wowww., Wonderful explanation.., never before ever after…
    True master for Big Data ,🙂Darshil

  • Akash Samanta, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Bhiya does data engineer mostly time work from home???

  • Vidya Nishok, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks and keep doing more… Please also make python part 2 ..

  • Roline John Aguilar, 1401-06-31 @ 11:08 ق.ظ پاسخ

    cool man, just what I needed

  • Lucas lopes, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Very good Darshil, keep the good work, cheers from Brasil.

  • Vivek Joshi, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hi Darshil!
    One small question. Sorry it's not related to DE but ETL.
    Do I need know ETL for a data analyst jobs in India ?
    I have looked Job desctiption of data analyst jobs in India and the US & I saw this pattern of "requiring knowledge of ETL & Data Warehouse" in most DA jobs in India where as DA jobs in the US have least number of requirements.

    I don't know what is the ground reality of Data Analyst jobs in India but things like those make me scare away from applying to DA jobs in the country until I get ETL knowledge(my brain tells me that). Now I don't know heck about ETL because my professional Certification did not cover ETL so I am unable to freely apply for jobs and feel chained due to not knowing that one ETL technology.
    Thank you if you could help.

  • Ram Gurung, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks!

  • keyur solanki, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Love the way you have explained things. Loads of success to you😇

  • Partab Parmar, 1401-06-31 @ 11:08 ق.ظ پاسخ

    THANK YOU SO MUCH FOR SUCHA QUALITY CONTENT ……GOD BLESS YOU

  • Khushnood Ahmed, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Great explanation! Will try to replicate in GCP

  • ARITRA, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Please take out time and create deeper conceptual projects like this. If possible please provide real time scenario based projects. We will be really interested to get further depth. This one is awesome!!

  • Rohan Kupate, 1401-06-31 @ 11:08 ق.ظ پاسخ

    mai ye begining karna chahtu hu par muze aws nhi ata aur Apache bhi nhi

  • HIGHLIGHTS WALA, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks a lot ❤️

  • Ariel Carambula, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hi Darshil I have this error following your code:

    453 – You currently have Essential access which includes access to Twitter API v2 endpoints only. If you need access to this endpoint, you’ll need to apply for Elevated access via the Developer Portal
    Thanks in advance

  • Meet Patel, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hii darshil
    Could you tell that will there be a job of data engineering in web 3.0 or not because data will be decentralised in web 3.0

  • Shubham Muley, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks Darshil for the project ! Loved it ! Can you suggest topics/ideas for more projects so that we can do it on our own ?

  • Venkata H, 1401-06-31 @ 11:08 ق.ظ پاسخ

    My sincere thanks to your efforts for contributing these kind of real time scenario based use case examples. 🙏

  • arnav roy, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Very knowledgeable

  • Ram Gupta, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hi Darshil, what's the pre requisite for this content. Any cloud or prior knowledge is required?

  • Mesum Raza Hemani, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Very good! Make more of such videos.

  • Abhijeet pradhan, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Best content ever..👍
    So nice of you..
    Really want more such Projects..

  • Dheerendra Negi, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Plz make a data engineering course.

  • payday_11, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Please keep making more videos like this!

  • Tejas Borkar, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Great darshil this is really helpful video
    Thank you.. keep uploading content like this please

  • shashank emani, 1401-06-31 @ 11:08 ق.ظ پاسخ

    This is amazing. Thank you! I have a question. Can we call extracting Twitter data as web scrapping?

  • Rider Comrade, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks

  • Abdul Razak Shaik, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Really Appreciate your efforts to the Data Community you are doing a great job. Thank you ❤️

  • JITENDRA PARIHAR, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Eagerly waiting for this project. Thank you darshil for such amazing projects…

  • Shivansh Pratap SP, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Make more and more projects.

  • sumanta ghosh, 1401-06-31 @ 11:08 ق.ظ پاسخ

    this is what is called content and actual do something for community ..rather than just doing faltu things…thank u so much Darshil ❤️

  • Rakesh Rout, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Can airflow run on windows ?

  • Shrikant Pandey, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Make more on airflow and end to end data pipeline. Great effort 🙂

  • Himanshu Chavan, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thank You Very Much for the project Darshil ❤️. Could you recommend any laptop for doing end to end data engineering projects? I am very confused about it. Please help me with this. Thank You once again.

  • Manish Tomar, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Please make a tutorial series on Airflow

  • aarthi, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks you… Waiting for this

  • Hritika Pal, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Hey thanks man I've been eagerly waiting for this. Would now explore and learn!

  • Aarav Singh, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanku darshil for making this project, please make this type of project on regular basis so that it will help us to enhance our skill

  • Aman, 1401-06-31 @ 11:08 ق.ظ پاسخ

    Thanks a lot Darshil for this , was eagerly waiting for it will be doing it today itself, Cant wait for many more of this kind of content. Please make more of these kind of DE projects, your youtube project is really a great project to work on and gain some real good insights.

  • karan gupta, 1401-06-31 @ 11:08 ق.ظ پاسخ

    this is perfect, thank you Darshil

دیدگاهتان را بنویسید

نشانی ایمیل شما منتشر نخواهد شد.