Goals achieved
Our collaborative effort resulted in the successful
creation of a user-friendly stock market analysis application
-
Effective Data Processing
Leveraging PySpark, we successfully processed and analyzed a massive dataset of over 150GB, sourced from Kaggle. This facilitated comprehensive insights into global stock markets.
-
Collaborative Development
Utilizing GitHub for version control, we established an efficient and collaborative development environment. This allowed seamless code synchronization among team members and ensured effective version management.
-
Versatile Execution Environments
We achieved the goal of providing users with the flexibility to execute the application both locally and on Google Cloud. Local execution involved setting up Python, JRE, and PySpark, while Google Cloud offered a scalable cloud environment with Dataproc clusters.
-
User-Friendly Interface
The implementation of a user-friendly interface in the main.py script allowed users to interact effortlessly with the application. The menu-driven system offered diverse options for exploring historical stock data, growth predictions, and probability estimations.
-
Diverse Analytical Capabilities
The application met the goal of providing a range of analytical options, including insights into historical stock data, growth predictions, and probability estimations for value increases in specific years. Users could explore expensive stocks historically, by year, or by country.
-
Dependency Management
We successfully managed dependencies using pip, ensuring smooth integration of PySpark and other essential components. This contributed to the stability and reliability of the application.
What did we learn?
During the process of making this application, we have learned how to create a software program using Python and PySpark. We have also learned how to use different functionalities from PySpark and run it on Google Cloud.
This project has also given us a lot of new insights about stocks from other countries that we did not know before. We appreciate all the valuable knowledge gained through this project, which has been really cool!
Future work
This project is the first approach to a financial advisor tool, we have only integrated 3 main functionalities, the estimation of stock growth, a view of the highest-valued stocks, and the prediction of the stock market in a year.
As further features or improvements for implementation, the stocks could be divided into industry sectors in order to give a better estimation of the stock market, and offer a more accurate perspective of the market to the user. This way we could also get the correlation of ups and downs between sectors.
Additionally, we could add important financial events to the dataset in order to explain changes in some stocks over time as well as integrating a GUI for a better user experience.