Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • sgraupner/ds_cs4bd_2324
  • akbi5459/ds_cs4bd_2324
  • s90907/ds_cs4bd_2324
  • fapr2511/ds_cs4bd_2324
4 results
Show changes
---
# ----------------------------------------------------------------------------------------
# -- Docs: https://github.com/cluster-apps-on-docker/spark-standalone-cluster-on-docker --
# ----------------------------------------------------------------------------------------
version: "3.6"
volumes:
shared-workspace:
name: "hadoop-distributed-file-system"
driver: local
services:
jupyterlab:
image: andreper/jupyterlab:3.0.0-spark-3.0.0
container_name: jupyterlab
ports:
- 8888:8888
- 4040:4040
- 4041:4041 # fail-over port when 4040 has error
volumes:
- shared-workspace:/opt/workspace
spark-master:
image: andreper/spark-master:3.0.0
container_name: spark-master
ports:
- 8080:8080
- 7077:7077
volumes:
- shared-workspace:/opt/workspace
spark-worker-1:
image: andreper/spark-worker:3.0.0
container_name: spark-worker-1
environment:
- SPARK_WORKER_CORES=1
- SPARK_WORKER_MEMORY=512m
ports:
- 8081:8081
volumes:
- shared-workspace:/opt/workspace
depends_on:
- spark-master
spark-worker-2:
image: andreper/spark-worker:3.0.0
container_name: spark-worker-2
environment:
- SPARK_WORKER_CORES=1
- SPARK_WORKER_MEMORY=512m
ports:
- 8082:8081
volumes:
- shared-workspace:/opt/workspace
depends_on:
- spark-master
...
\ No newline at end of file
......@@ -4,8 +4,33 @@ This repository contains the assignments for class
[Computer Science for Big Data](https://lms.bht-berlin.de/course/view.php?id=30704):
- Assignment A - [Python setup](A_setup_python).
- Assignment B - [Explore Python](B_explore_python).
- Assignment C - [Python Expressions & Unit Tests](C_expressions).
- Assignment D - [Recursive Problem Solving](D_recursion).
- Assignment E - [Data Streams](E_streams).
- Assignment F - [Graph Data](F_graph_data).
- Assignment G - [Docker](G_docker).
- Assignment DB - Building a
[SQL Database](https://github.com/sgra64/db-freerider) with tasks:
- [DB1](https://github.com/sgra64/db-freerider/blob/main/markup/Ue_DB1_Server_Build.md)
(DB-Server Build with Docker) and
- [DB2](https://github.com/sgra64/db-freerider/blob/main/markup/Ue_DB2_Schema_Data_Build.md)
(DB Build with DB-Schema and Data load, Queries).
- Assignment H - [Apache Spark](H_Spark).
- Assignment H1 - [Shakespeare](H_Spark/H1_Shakespeare.md) with Apache Spark.
 
Follow the [link](https://docs.google.com/spreadsheets/d/1kDqVq45LQGol8Lskz3mra622QcFro9RBx-U2gI2U3hI/edit#gid=0)
......
markup/img/C_unit_tests_1.png

187 KiB

markup/img/C_unit_tests_2.png

157 KiB

markup/img/C_unit_tests_3.png

160 KiB

markup/img/C_unit_tests_4.png

223 KiB

markup/img/G_docker_img01.png

141 KiB

markup/img/G_docker_img02.png

62.9 KiB

markup/img/G_docker_img03.png

98.1 KiB

markup/img/H_img01.png

100 KiB

markup/img/H_img02.png

112 KiB

markup/img/H_spark_cluster_architecture.png

33.5 KiB

markup/img/fibonacci.jpg

50.5 KiB

markup/img/graph_1.jpg

35 KiB

markup/img/graph_2a.jpg

29 KiB

markup/img/graph_2b.jpg

27.6 KiB