{"bda\/2\/2\/b":{"title":"BDA2.2 Jupyter Notebooks","id":"bda\/2\/2\/b","requirements":[],"learning outcomes":["* **Understand the architecture** of Jupyter Notebooks and their integration within HPC environments to support scalable data analysis.","* **Create and manage Jupyter Notebook instances** in local and remote settings, leveraging HPC resources.","* **Develop and execute code** efficiently in Notebooks using languages like Python, R, and Julia.","* **Utilize interactive widgets** to create dynamic, user-interactive data visualizations and tools within Notebooks.","* **Integrate Jupyter Notebooks with big data tools** like Apache Spark to handle large datasets effectively.","* **Employ advanced visualization libraries** within Notebooks to represent complex datasets visually.","* **Collaborate effectively using Jupyter Notebooks**, sharing notebooks for reproducible research and peer review.","* **Optimize Notebook performance** in big data applications, including memory management and parallel computing techniques.","* **Implement best practices for security** in Jupyter Notebooks, securing sensitive data and computational processes.","* **Explore extensions and plugins** that enhance the functionality of Jupyter Notebooks, such as dashboarding tools and version control integration.","* **Analyze case studies** where Jupyter Notebooks have been effectively used in big data projects across various industries.","* **Participate in hands-on workshops** to gain practical experience with complex analytical tasks using Jupyter Notebooks.","* **Navigate ethical and legal considerations** in using Jupyter Notebooks, especially regarding data privacy and usage rights in collaborative environments.","* **Stay updated with the latest developments** in the Jupyter ecosystem, including new tools and features that can enhance data analysis workflows.","AI generated content"],"exams":{"questions":0}},"bda\/2\/b":{"title":"BDA2 Big Data Tools in HPC","id":"bda\/2\/b","learning outcomes":["* Distinguish the benefit and drawback of various big data tools in the HPC environment including","* Ophidia","* Jupyter Notebook","* RayDP","* Spark_horovod","* Dask","* Apply a data science workflow on existing data using various big data tools.","* Construct simple data science workflows.","* Analyze the use of cloud computing on Big Data"],"subskills":["bda\/2\/1\/b","bda\/2\/2\/b","bda\/2\/3\/b","bda\/2\/4\/b","bda\/2\/5\/b","bda\/2\/6\/b"],"exams":{"questions":0}},"bda\/2\/5\/b":{"title":"BDA2.5 Spark-Horovod","id":"bda\/2\/5\/b","requirements":[],"learning outcomes":["* **Understand the integration of Spark with Horovod**, recognizing how it facilitates distributed deep learning.","* **Set up and configure Spark-Horovod environments** in HPC settings, ensuring optimal configuration for specific project requirements.","* **Execute distributed training sessions** using Spark-Horovod, applying it to practical machine learning problems involving large datasets.","* **Optimize data preprocessing tasks** within Spark to feed into Horovod-powered training pipelines efficiently.","* **Leverage Spark\u2019s data handling capabilities** to manage the input and output of machine learning models trained with Horovod.","* **Utilize advanced features of Horovod** such as gradient aggregation and checkpointing to improve the efficiency of model training.","* **Develop scalable machine learning applications** that combine Spark\u2019s big data processing with Horovod\u2019s efficient computation distribution.","* **Monitor and debug distributed training processes**, using tools integrated within Spark and Horovod to track performance and identify bottlenecks.","* **Explore case studies** demonstrating the successful application of Spark-Horovod in industries such as finance, healthcare, and retail.","* **Participate in hands-on labs** to experience real-world challenges and solutions in training deep learning models at scale.","* **Navigate data governance and security issues** in distributed machine learning environments, focusing on compliance and data protection.","* **Assess the performance and scalability** of Spark-Horovod, comparing it to other distributed machine learning frameworks.","* **Discuss future trends and advancements** in the integration of big data processing and distributed deep learning.","* **Collaborate on projects using Spark-Horovod**, fostering teamwork and knowledge exchange among peers.","* **Critically evaluate the impact of using Spark-Horovod** on machine learning project outcomes, focusing on improvements in speed, scalability, and accuracy.","* **Implement post-training tasks** such as model evaluation and deployment within the Spark-Horovod ecosystem.","* **Analyze resource allocation and management** within Spark-Horovod to optimize computational efficiency.","* **Integrate Spark-Horovod with other big data platforms** and ecosystems to enhance data flow and processing capabilities.","* **Design resilient and fault-tolerant systems** using Spark-Horovod to handle failures in distributed environments.","* **Advance knowledge in tuning hyperparameters** within distributed settings to maximize model performance.","AI generated content"],"exams":{"questions":0}},"bda\/2\/1\/b":{"title":"BDA2.1 Ophidia","id":"bda\/2\/1\/b","requirements":[],"learning outcomes":["* **Understand Ophidia\u2019s architecture** and how it integrates with existing HPC environments to support scalable data analytics.","* **Learn to use Ophidia's array-based data model** for efficient data storage, retrieval, and processing.","* **Implement data operations** using Ophidia's functional interface, which includes aggregation, selection, and array manipulation.","* **Develop workflows using Ophidia\u2019s workflow management tools** to automate and optimize complex data analysis tasks.","* **Utilize Ophidia's large-scale analytical operators** to perform high-level, scientific data analyses and transformations.","* **Explore case studies** that demonstrate the application of Ophidia in real-world scientific research, particularly in climate and environmental data analysis.","* **Integrate visualization tools** with Ophidia to create insightful graphical representations of large-scale data sets.","* **Assess the performance benefits** of using Ophidia in big data projects, comparing it with other analytics tools in terms of speed and scalability.","* **Navigate data privacy and security considerations** in the context of using Ophidia for sensitive or proprietary scientific data.","* **Participate in hands-on labs** to gain practical experience with Ophidia, focusing on setup, configuration, and execution of typical data workflows.","* **Critically analyze** the suitability of Ophidia for various types of data-intensive applications in scientific research.","* **Master the use of Ophidia\u2019s built-in functions** for complex statistical analysis.","* **Implement advanced data reduction techniques** to manage large datasets effectively.","* **Develop custom operators** for tailored scientific analysis.","* **Explore parallel data processing capabilities** to improve computational efficiency.","* **Apply multidimensional data analysis** across varied scientific domains.","* **Understand metadata management** to optimize data discovery and retrieval.","* **Investigate interoperability** with other big data tools and formats.","* **Learn about data provenance** to enhance reproducibility.","* **Engage in collaborative projects** to tackle complex challenges using Ophidia.","* **Explore real-time data processing capabilities** for streaming data scenarios.","AI generated content"],"exams":{"questions":0}},"bda\/2\/6\/b":{"title":"BDA2.6 Dask","id":"bda\/2\/6\/b","requirements":["* Familiarity with Python programming","* Understanding of NumPy and pandas","* Basic knowledge of parallel computing concepts"],"learning outcomes":["* **Understands what Dask is** and when to use it","* **Understands the purpose of Dask** and its benefits in parallelizing workflows","* **Distinguishes between Dask collections** (arrays, dataframes, bags)","* **Converts existing NumPy\/pandas code** to use Dask","* **Run Dask workloads locally** using the default scheduler","* **Writes a small Dask script** that uses dask.delayed","* **Uses the Dask dashboard** to monitor Dask computations and visualize task executions","* **Compares the runtime** of equivalent Dask and NumPy\/pandas computations on large data","AI generated content"],"exams":{"questions":0}},"bda\/2\/3\/b":{"title":"BDA2.3 Cloud","id":"bda\/2\/3\/b","requirements":[],"learning outcomes":["* **Understand the role of cloud computing** in big data analytics, identifying how cloud resources can be leveraged to process and analyze large datasets.","* **Explore various cloud services** and solutions that support big data projects, including data storage, computation, and analytics platforms.","* **Implement data migration strategies** to efficiently transfer large volumes of data to and from cloud environments.","* **Utilize cloud-based HPC solutions** to perform scalable and efficient data analyses, examining the trade-offs between on-premise and cloud-based HPC.","* **Develop and deploy applications** in the cloud using containerization and orchestration tools like Docker and Kubernetes.","* **Optimize cost and resource usage** in cloud environments, employing best practices for cloud resource management and cost-efficiency.","* **Secure cloud-based data solutions**, understanding security best practices and compliance issues related to data in the cloud.","* **Integrate cloud data services with existing HPC infrastructure**, ensuring seamless data flow and maintenance.","* **Evaluate the performance of cloud services** using benchmarks and performance metrics specific to big data applications.","* **Participate in hands-on labs** to set up and configure cloud environments for real-world data analytics scenarios.","* **Assess the scalability and elasticity** of cloud solutions, understanding how to dynamically adjust resources based on workload demands.","* **Navigate ethical and legal considerations** of storing and processing data in the cloud, especially in a multi-tenant environment.","* **Explore innovative cloud technologies** and emerging trends that influence big data analytics, such as serverless computing and machine learning services.","* **Collaborate across distributed teams** using cloud-based tools and platforms to enhance productivity and data sharing in big data projects.","* **Analyze the impact of cloud computing on data governance** and regulatory compliance, focusing on data sovereignty and auditability.","* **Explore advanced networking configurations** for cloud environments to enhance data transfer speeds and reduce latency in big data processing.","* **Understand the use of APIs in cloud environments** to automate tasks and integrate diverse data sources and analytical tools.","* **Implement disaster recovery and data backup strategies** in the cloud to ensure data integrity and availability.","* **Develop multi-cloud strategies** to avoid vendor lock-in and enhance resilience in big data analytics.","* **Utilize AI and machine learning workflows** in the cloud to automate data analysis and generate insights at scale.","AI generated content"],"exams":{"questions":0}},"bda\/2\/4\/b":{"title":"BDA2.4 RayDP","id":"bda\/2\/4\/b","requirements":[],"learning outcomes":["* **Understand RayDP\u2019s architecture** and its integration points with Apache Spark, identifying how it enhances functionality in distributed data processing.","* **Set up and configure RayDP** in a HPC environment, optimizing it for specific data workflows.","* **Execute large-scale data processing tasks** using RayDP, combining Ray\u2019s scalability with Spark\u2019s data processing tools.","* **Develop machine learning pipelines** utilizing RayDP, integrating Spark MLlib and Ray\u2019s machine learning libraries.","* **Optimize data operations** with RayDP for improved performance and efficiency in data handling and computation.","* **Handle streaming data** using RayDP\u2019s integration with Spark Streaming, processing real-time data feeds effectively.","* **Implement complex analytics workflows** that leverage the full capabilities of both Ray and Spark through RayDP.","* **Utilize RayDP for AI and deep learning tasks**, taking advantage of Ray\u2019s support for scalable model training and inference.","* **Explore the use of RayDP in various industry sectors**, such as finance, healthcare, and telecommunications, for real-world applications.","* **Participate in hands-on labs** to gain practical experience with deploying, managing, and optimizing RayDP-based applications.","* **Navigate the scalability and resource management challenges** in RayDP, applying best practices for large-scale data deployments.","* **Assess the impact of RayDP on project outcomes**, analyzing improvements in processing times and resource utilization.","* **Explore the future potential of RayDP**, discussing upcoming features and potential enhancements in the integration of Ray and Spark.","* **Address security considerations and data governance** in RayDP applications, ensuring compliance with legal and ethical standards.","* **Collaborate effectively in distributed teams** using RayDP, enhancing communication and project management in big data projects.","* **Critically evaluate RayDP\u2019s performance** against other big data frameworks, understanding its unique advantages and limitations.","AI generated content"],"exams":{"questions":0}},"bda\/b":{"title":"BDA Big Data Analytics","id":"bda\/b","learning outcomes":["* Examine some theoretical principles of big data.","* Differentiate the various tools that could be used in an HPC environment effectively.","* Contrast the different steps in data processing from preparation, to post-processing, and visualization and analysis.","* Describe and apply the concepts of artificial intelligence and data science.","* Compose a workflow consisting of HPC and big data tools to analyze the data.","* Apply statistical evaluation techniques to assess the generalization and reliability of ML models and benchmark AI models at scale with a focus on consistency, fairness, and comparability."],"subskills":["bda\/1\/b","bda\/2\/b","bda\/3\/b","bda\/4\/b","bda\/5\/b","bda\/6\/b","bda\/6\/b","bda\/7\/b"],"exams":{"questions":0}},"bda\/5\/2\/2\/b":{"title":"BDA5.2.2 Neural Network Architectures","id":"bda\/5\/2\/2\/b","requirements":["* External: Understanding of neural network basics","* Internal: BDA5.2.1 Basics of Neural Networks (recommended)"],"learning outcomes":["* Describe the architecture and applications of convolutional neural networks (CNNs).","* Explain how recurrent neural networks (RNNs), LSTMs, and GRUs process sequential data.","* Understand the transformer architecture and its impact on modern AI models.","* Compare trade-offs among architectures in terms of complexity, performance, and parallelism.","* Select appropriate architectures based on task type and data modality.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/2\/b":{"title":"BDA5.2 Deep Learning Fundamentals","id":"bda\/5\/2\/b","learning outcomes":["* Describe how neural networks are structured, trained, and applied in data-driven tasks.","* Differentiate between core deep learning architectures such as CNNs, RNNs, and transformers."],"subskills":["bda\/5\/2\/1\/b","bda\/5\/2\/2\/b"],"exams":{"questions":0}},"bda\/5\/2\/1\/b":{"title":"BDA5.2.1 Basics of Neural Networks","id":"bda\/5\/2\/1\/b","requirements":["* External: Knowledge of basic linear algebra and calculus","* Internal: None"],"learning outcomes":["* Describe the structure of a feedforward neural network.","* Explain the role of activation functions and compare common types (ReLU, sigmoid, tanh).","* Understand how neural networks learn through backpropagation and gradient descent.","* Identify common loss functions for classification and regression tasks.","* Outline the training loop and how it is implemented in modern frameworks.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/b":{"title":"BDA5 Machine Learning","id":"bda\/5\/b","learning outcomes":["* Explain the fundamental differences between supervised, unsupervised, and deep learning approaches.","* Interpret evaluation metrics for different model types and assess performance quality.","* Describe how neural networks are structured and how architecture choice affects modeling tasks.","* Use major ML\/DL frameworks such as PyTorch and TensorFlow to implement models.","* Apply distributed training techniques to scale model development across HPC resources.","* Tune performance using batch sizing, mixed precision, and checkpointing for long-running jobs.","* Design and execute hyperparameter search experiments efficiently at scale.","* Implement resource-aware tuning strategies that consider runtime, memory, and energy trade-offs.","* Integrate optimization, monitoring, and reproducibility practices for ML workflows on HPC platforms."],"subskills":["bda\/5\/1\/b","bda\/5\/2\/b","bda\/5\/3\/b","bda\/5\/4\/b","bda\/5\/5\/b"],"exams":{"questions":0}},"bda\/5\/5\/2\/b":{"title":"BDA5.5.2 Distributed Hyperparameter Search","id":"bda\/5\/5\/2\/b","requirements":["* External: Understanding of hyperparameter tuning strategies and batch job submission","* Internal: BDA5.5.1 Hyperparameter Search Methods (recommended)"],"learning outcomes":["* Explain the benefits and challenges of distributing hyperparameter search in HPC environments.","* Use orchestration tools such as Ray Tune, Optuna, or Hyperopt for parallel execution.","* Integrate distributed tuning with HPC schedulers like Slurm or PBS.","* Monitor, resume, and manage many concurrent tuning jobs.","* Optimize compute resource usage to balance speed, cost, and coverage.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/5\/b":{"title":"BDA5.5 Hyperparameter Tuning","id":"bda\/5\/5\/b","learning outcomes":["* Apply search algorithms to explore hyperparameter spaces effectively.","* Scale tuning processes across HPC resources while minimizing overhead and waste."],"subskills":["bda\/5\/5\/1\/b","bda\/5\/5\/2\/b","bda\/5\/5\/3\/b","bda\/5\/5\/4\/b"],"exams":{"questions":0}},"bda\/5\/5\/1\/b":{"title":"BDA5.5.1 Hyperparameter Search Methods","id":"bda\/5\/5\/1\/b","requirements":["* External: Familiarity with model training and tuning concepts","* Internal: None"],"learning outcomes":["* Describe the role of hyperparameters in machine learning model performance.","* Compare manual tuning, grid search, and random search techniques.","* Explain how Bayesian optimization and other probabilistic methods guide search using prior results.","* Evaluate search efficiency and effectiveness across multiple performance metrics.","* Apply reproducible tuning experiments using configuration management tools or frameworks.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/5\/3\/b":{"title":"BDA5.5.3 Early Stopping and Pruning Strategies","id":"bda\/5\/5\/3\/b","requirements":["* External: Experience with model evaluation and validation techniques","* Internal: BDA5.5.1 Hyperparameter Search Methods (recommended)"],"learning outcomes":["* Define early stopping and its role in improving training efficiency.","* Describe criteria such as validation loss plateaus or metric stagnation.","* Apply pruning strategies to terminate weak candidates during hyperparameter search.","* Integrate early stopping with tuning frameworks and logging systems.","* Evaluate trade-offs between training completeness and resource savings.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/5\/4\/b":{"title":"BDA5.5.4 Resource Aware Tuning","id":"bda\/5\/5\/4\/b","requirements":["* External: Basic understanding of HPC scheduling and tuning processes","* Internal: BDA5.5.1 Hyperparameter Search Methods (recommended)"],"learning outcomes":["* Explain how resource availability and cost influence tuning strategy selection.","* Use adaptive or multi-fidelity tuning approaches to allocate compute efficiently.","* Apply constraints on runtime, memory, or energy usage in tuning campaigns.","* Integrate resource metrics into scheduling or pruning decisions.","* Evaluate tuning success with respect to both model performance and resource efficiency.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/1\/2\/b":{"title":"BDA5.1.2 Evaluation Metrics","id":"bda\/5\/1\/2\/b","requirements":["* External: Familiarity with supervised and unsupervised learning concepts","* Internal: BDA5.1.1 Supervised and Unsupervised Learning (recommended)"],"learning outcomes":["* Identify common metrics for classification (e.g., accuracy, precision, recall, F1-score) and regression (e.g., RMSE, MAE).","* Explain confusion matrices, ROC curves, and AUC for classification tasks.","* Describe cluster evaluation metrics like silhouette score and Davies-Bouldin index.","* Interpret metric values to assess underfitting, overfitting, and model calibration.","* Apply cross-validation to estimate model performance and generalizability.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/1\/b":{"title":"BDA5.1 Machine Learning Fundamentals","id":"bda\/5\/1\/b","learning outcomes":["* Explain the principles of supervised and unsupervised learning in data-driven tasks.","* Identify key metrics used to evaluate classification and regression models."],"subskills":["bda\/5\/1\/1\/b","bda\/5\/1\/2\/b"],"exams":{"questions":0}},"bda\/5\/1\/1\/b":{"title":"BDA5.1.1 Supervised and Unsupervised Learning","id":"bda\/5\/1\/1\/b","requirements":["* External: Basic statistics and data analysis knowledge","* Internal: None"],"learning outcomes":["* Define and distinguish between supervised and unsupervised learning paradigms.","* Identify common algorithms such as decision trees, SVMs, k-means, and PCA.","* Explain how labeled and unlabeled datasets are used in each approach.","* Describe typical workflows and applications of each paradigm in scientific and industrial settings.","* Understand data partitioning, overfitting, and model generalization in supervised learning.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/3\/2\/b":{"title":"BDA5.3.2 Tensorflow","id":"bda\/5\/3\/2\/b","requirements":["* External: Experience with Python and basic deep learning workflows","* Internal: BDA5.2.1 Basics of Neural Networks (recommended)"],"learning outcomes":["* Build and train models using TensorFlow's Keras API.","* Explain the difference between eager execution and static graph mode.","* Use tf.data pipelines for scalable and efficient data input.","* Implement model checkpoints and callbacks for training control.","* Apply GPU acceleration, distribution strategies, and TensorBoard for visualization.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/3\/b":{"title":"BDA5.3 ML and DL Frameworks","id":"bda\/5\/3\/b","learning outcomes":["* Use leading ML\/DL frameworks to define, train, and evaluate models.","* Apply distributed training strategies using framework-native tools for scalability and performance."],"subskills":["bda\/5\/3\/1\/b","bda\/5\/3\/2\/b","bda\/5\/3\/3\/b"],"exams":{"questions":0}},"bda\/5\/3\/1\/b":{"title":"BDA5.3.1 Pytorch","id":"bda\/5\/3\/1\/b","requirements":["* External: Programming experience with Python","* Internal: BDA5.2.1 Basics of Neural Networks (recommended)"],"learning outcomes":["* Define and train neural networks using the PyTorch nn.Module interface.","* Implement forward passes, loss computation, and backward propagation.","* Use DataLoader and Dataset classes to efficiently handle input data.","* Apply GPU acceleration and mixed precision with PyTorch utilities.","* Debug models interactively using PyTorch\u2019s dynamic computation graph.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/3\/3\/b":{"title":"BDA5.3.3 Distributed Training","id":"bda\/5\/3\/3\/b","requirements":["* External: Understanding of model training workflows","* Internal: BDA5.3.1 Pytorch or BDA5.3.2 Tensorflow (recommended)"],"learning outcomes":["* Distinguish between data parallelism and model parallelism strategies.","* Use PyTorch\u2019s DistributedDataParallel and TensorFlow\u2019s tf.distribute APIs.","* Configure multi-GPU and multi-node training in HPC environments.","* Monitor and debug performance bottlenecks in distributed training setups.","* Apply best practices for checkpointing, fault tolerance, and reproducibility at scale.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/4\/2\/b":{"title":"BDA5.4.2 Mixed Precision Training","id":"bda\/5\/4\/2\/b","requirements":["* External: Basic understanding of floating point computation","* Internal: BDA5.3.1 Pytorch or BDA5.3.2 Tensorflow (recommended)"],"learning outcomes":["* Define mixed precision training and describe its benefits in performance and memory usage.","* Identify hardware and software prerequisites for mixed precision support (e.g., NVIDIA Tensor Cores, AMP).","* Apply automatic mixed precision (AMP) in PyTorch and TensorFlow workflows.","* Monitor for numerical instability and apply scaling techniques as needed.","* Benchmark training speed and accuracy trade-offs using mixed precision.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/4\/b":{"title":"BDA5.4 HPC Optimization for ML","id":"bda\/5\/4\/b","learning outcomes":["* Optimize batch sizes and parallelism settings to improve training scalability.","* Apply mixed precision techniques and implement robust checkpointing strategies for long-running jobs."],"subskills":["bda\/5\/4\/1\/b","bda\/5\/4\/2\/b","bda\/5\/4\/3\/b"],"exams":{"questions":0}},"bda\/5\/4\/1\/b":{"title":"BDA5.4.1 Batch Size and Data Parallelism","id":"bda\/5\/4\/1\/b","requirements":["* External: Familiarity with model training and GPU compute","* Internal: BDA5.3.3 Distributed Training (recommended)"],"learning outcomes":["* Explain how batch size affects training stability, convergence, and throughput.","* Identify the relationship between batch size and memory usage on accelerators.","* Apply data parallelism techniques across GPUs or nodes for scalable training.","* Use gradient accumulation to simulate large batch sizes under memory constraints.","* Evaluate performance trade-offs using throughput and loss convergence metrics.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/5\/4\/3\/b":{"title":"BDA5.4.3 Checkpointing and Recovery","id":"bda\/5\/4\/3\/b","requirements":["* External: Familiarity with training loops and storage systems","* Internal: None"],"learning outcomes":["* Explain the importance of checkpointing for resiliency in HPC training workflows.","* Implement model, optimizer, and scheduler state saving in popular ML frameworks.","* Choose checkpointing frequency based on job length, stability, and system load.","* Manage checkpoint file size, compression, and storage placement.","* Integrate checkpointing with job resubmission and monitoring tools in HPC environments.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/1\/2\/b":{"title":"BDA1.2 AI and Data Science","id":"bda\/1\/2\/b","requirements":[],"learning outcomes":["* **Understand the fundamental concepts** behind AI and its role in data science.","* **Explore various AI methodologies** that enhance data analysis, including machine learning, neural networks, and deep learning.","* **Apply AI techniques** to automate data processing, model complex patterns, and predict future trends based on big data sets.","* **Integrate statistical methods** with AI tools to validate data models and ensure the accuracy of predictions.","* **Utilize advanced analytics software** and platforms to conduct data science projects efficiently.","* **Develop skills in data preprocessing**, transformation, and cleaning to prepare data for AI-driven analysis.","* **Implement real-world AI projects** to solve business problems using data science principles.","* **Evaluate the effectiveness** of AI models in real-time applications across different sectors such as finance, healthcare, and technology.","* **Critically assess ethical and privacy issues** surrounding AI in the context of data science.","* **Discuss case studies** highlighting successful AI applications in data science.","* **Navigate the challenges** of integrating AI with traditional data analysis techniques.","* **Enhance decision-making processes** by incorporating AI-driven insights into business strategies.","* **Lead teams in collaborative data projects**, fostering a culture that embraces AI innovations.","* **Stay updated with the latest advancements** in AI technologies and their implications for data science.","* **Develop comprehensive documentation** for AI projects, including design, implementation strategies, and outcomes.","* **Facilitate workshops or training sessions** to disseminate knowledge on AI applications in data science.","* **Analyze the scalability** of AI solutions in handling increasingly large and complex data sets.","* **Explore specialized AI fields** such as natural language processing (NLP) and computer vision for specific data science applications.","* **Craft robust data governance frameworks** to manage AI systems effectively.","* **Participate in professional networks** to exchange knowledge and collaborate on emerging AI data science technologies.","* **Investigate the use of AI for predictive analytics** and its impact on various industries.","* **Explore machine learning pipelines** from data ingestion to model deployment and monitoring.","* **Understand the impact of AI on data quality, storage, and retrieval processes**.","* **Develop problem-solving skills** to tackle complex analytical challenges using AI.","* **Foster an understanding of the interplay between big data infrastructures and AI operations** to optimize performance and resource management.","AI generated content"],"exams":{"questions":0}},"bda\/1\/b":{"title":"BDA1 Theoretic Principles of BDA","id":"bda\/1\/b","learning outcomes":["* Describe the key concepts of artificial intelligence and data science.","* Describe the general approach of big data tools.","* Illustrate the typical data science workflow.","* Categorize the different types of diagrams used for visualization.","* List ethical constraints.","**AI generated content**"],"subskills":["bda\/1\/1\/b","bda\/1\/2\/b","bda\/1\/3\/b","bda\/1\/4\/b","bda\/1\/5\/b"],"exams":{"questions":0}},"bda\/1\/5\/b":{"title":"BDA1.5 Ethical\/Privacy","id":"bda\/1\/5\/b","requirements":[],"learning outcomes":["* **Understand the ethical implications** of big data analytics on individuals, communities, and society.","* **Examine privacy regulations** such as GDPR, CCPA, and HIPAA and their implications for big data analytics.","* **Analyze ethical dilemmas** related to data collection, storage, analysis, and dissemination in big data projects.","* **Discuss the ethical responsibilities** of data scientists, analysts, and organizations in ensuring fairness, transparency, and accountability.","* **Explore the role of bias** in data collection, algorithmic decision-making, and model outcomes, and strategies for bias mitigation.","* **Understand the principles of data anonymization** and pseudonymization for protecting individual privacy while preserving data utility.","* **Analyze the impact of data breaches** and security vulnerabilities on individual privacy, organizational reputation, and regulatory compliance.","* **Discuss ethical considerations** in data sharing and collaboration, including informed consent, data ownership, and intellectual property rights.","* **Explore the ethical use of predictive analytics** in sensitive domains such as healthcare, criminal justice, and finance, and potential biases and risks.","* **Examine the ethical implications** of surveillance technologies, facial recognition systems, and biometric data collection in public and private settings.","* **Discuss the ethical challenges** of algorithmic transparency, explainability, and accountability in automated decision-making systems.","* **Analyze the ethical implications** of data-driven automation and artificial intelligence in reshaping labor markets, job displacement, and socioeconomic inequalities.","* **Explore ethical considerations** in the use of data-driven technologies for political campaigning, voter targeting, and manipulation of public opinion.","* **Discuss the ethical dimensions** of data-driven approaches to environmental monitoring, climate change mitigation, and sustainable development.","* **Analyze the ethical implications** of data-driven approaches to public health surveillance, disease monitoring, and pandemic response, including issues of privacy and consent.","AI generated content"],"exams":{"questions":0}},"bda\/1\/1\/b":{"title":"BDA1.1 6Vs","id":"bda\/1\/1\/b","requirements":[],"learning outcomes":["* **Define and understand the 6Vs** of big data: Volume, Velocity, Variety, Veracity, Value, and Variability.","* **Analyze the impact of Volume** on data storage and processing infrastructure.","* **Explore Velocity** in the context of real-time data processing and decision-making.","* **Assess the challenges and strategies** related to managing Variety in data types and sources.","* **Evaluate data Veracity** and its influence on the accuracy and trustworthiness of analytical insights.","* **Determine the Value** derived from data analytics and how it drives business and organizational strategy.","* **Examine Variability** and its effects on data consistency and interpretation.","* **Apply techniques to handle the 6Vs** in practical data analysis scenarios using modern tools and technologies.","* **Discuss case studies** that illustrate the real-world application of the 6Vs in various industries.","* **Develop strategies for optimizing data collection and analysis** processes to maximize the benefits of the 6Vs.","* **Create metrics to measure the impact of the 6Vs** on the performance and success of big data projects.","* **Explore technological solutions** like cloud computing and distributed databases that support managing the 6Vs.","* **Conduct workshops or seminars** to foster understanding and application of the 6Vs in organizational contexts.","* **Critique current methodologies** in managing the 6Vs and propose improvements based on emerging technologies.","* **Simulate scenarios** to practice the application of theoretical knowledge about the 6Vs in solving business problems.","* **Discuss ethical and legal considerations** in managing and analyzing big data according to the 6Vs.","* **Lead collaborative projects** that enhance practical skills in managing the complexities of big data.","* **Stay updated with the latest trends** and technologies in big data that influence the 6Vs.","* **Develop comprehensive documentation** for managing big data projects with a focus on the 6Vs.","* **Engage with experts** through guest lectures or industry partnerships to gain insights into advanced practices related to the 6Vs.","* **Integrate data governance and compliance** practices into big data projects affected by the 6Vs.","* **Design interactive visualizations** to represent data affected by each of the 6Vs, enhancing data interpretation and decision-making.","* **Utilize advanced analytics techniques** to extract actionable insights from data characterized by the 6Vs.","* **Explore the interrelationships between the 6Vs** to understand how they influence each other and impact overall data management strategies.","* **Foster a data-driven culture** within organizations by emphasizing the importance and strategic value of managing the 6Vs effectively.","AI generated content"],"exams":{"questions":0}},"bda\/1\/3\/b":{"title":"BDA1.3 Data Mining","id":"bda\/1\/3\/b","requirements":[],"learning outcomes":["* **Understand the fundamentals** of data mining and its role in extracting valuable insights from large datasets.","* **Explore different data mining techniques** such as classification, clustering, association rule mining, and anomaly detection.","* **Apply data preprocessing methods** to clean, transform, and normalize raw data for effective mining.","* **Implement classification algorithms** including decision trees, logistic regression, support vector machines (SVM), and k-nearest neighbors (KNN).","* **Utilize clustering algorithms** such as k-means, hierarchical clustering, and density-based clustering for unsupervised learning tasks.","* **Analyze association rule mining** techniques like Apriori algorithm for discovering interesting relationships among data items.","* **Detect anomalies** in datasets using statistical methods and machine learning algorithms.","* **Evaluate the performance** of data mining models using metrics like accuracy, precision, recall, and F1-score.","* **Apply ensemble learning techniques** to improve the robustness and accuracy of data mining models.","* **Explore advanced topics** in data mining such as text mining, web mining, and social network analysis.","* **Understand the ethical implications** of data mining, including privacy concerns and algorithmic bias.","* **Develop data mining pipelines** for end-to-end analysis, from data collection to model deployment.","* **Integrate data mining tools** with big data platforms such as Apache Hadoop and Spark for scalable processing.","* **Analyze real-world case studies** to understand the practical applications of data mining in various domains.","* **Optimize data mining algorithms** for performance and scalability in distributed computing environments.","* **Discuss the challenges** and limitations of data mining techniques, including handling noisy and incomplete data.","* **Participate in data mining competitions** to apply learned techniques and solve real-world problems.","* **Stay updated with the latest advancements** in data mining research and industry trends.","* **Collaborate with domain experts** to interpret data mining results and derive actionable insights.","* **Communicate findings effectively** through data visualization and storytelling techniques.","* **Explore the use of data mining in business intelligence** for strategic decision-making and competitive analysis.","* **Develop critical thinking skills** to formulate hypotheses and design experiments for data mining tasks.","* **Understand the role of feature engineering** in improving the performance of data mining models.","* **Analyze the impact of data quality** on the effectiveness of data mining processes.","* **Explore data mining in streaming and real-time analytics** environments for timely decision-making.","* **Investigate techniques for handling imbalanced datasets** to improve the performance of classification models.","* **Examine approaches for interpretability and explainability** of data mining models to enhance trust and understanding.","* **Explore techniques for scalability and parallelization** of data mining algorithms to handle large-scale datasets efficiently.","* **Discuss strategies for data anonymization and de-identification** to protect sensitive information while maintaining utility for analysis.","* **Investigate techniques for sequential pattern mining** to uncover temporal relationships and patterns in sequential data.","AI generated content"],"exams":{"questions":0}},"bda\/1\/4\/b":{"title":"BDA1.4 Algorithms","id":"bda\/1\/4\/b","requirements":[],"learning outcomes":["* **Understand the role of algorithms** in solving computational problems and optimizing data processing tasks.","* **Explore different algorithm paradigms** including divide and conquer, dynamic programming, greedy algorithms, and randomized algorithms.","* **Analyze the time and space complexity** of algorithms using Big O notation to assess their efficiency and scalability.","* **Implement graph algorithms** such as breadth-first search (BFS), depth-first search (DFS), Dijkstra's algorithm, and minimum spanning tree (MST) algorithms.","* **Apply optimization techniques** such as linear programming, integer programming, and network flow algorithms to solve optimization problems in big data analytics.","* **Utilize string matching algorithms** including exact matching (e.g., KMP algorithm) and approximate matching (e.g., Levenshtein distance) for text processing tasks.","* **Implement sorting and searching algorithms** such as quicksort, mergesort, binary search, and hash-based searching techniques for efficient data retrieval.","* **Analyze algorithmic trade-offs** between time complexity, space complexity, and implementation simplicity in various big data scenarios.","* **Explore parallel and distributed algorithms** for efficient processing of large-scale datasets across distributed computing environments.","* **Apply machine learning algorithms** such as linear regression, logistic regression, decision trees, and clustering algorithms for predictive analytics tasks.","* **Understand the principles of approximation algorithms** and their applications in solving NP-hard optimization problems in big data analytics.","* **Analyze the impact of algorithmic bias** on decision-making processes and strategies for mitigating bias in algorithm design.","* **Explore online algorithms** for processing streaming data and making real-time decisions in dynamic environments.","* **Investigate probabilistic algorithms** including Monte Carlo methods and randomized algorithms for uncertainty quantification and simulation tasks.","* **Apply graph processing algorithms** such as PageRank, community detection algorithms, and graph neural networks for analyzing complex networks and social graphs.","* **Examine distributed consensus algorithms** such as Paxos and Raft for achieving fault tolerance and consistency in distributed systems.","* **Explore advanced topics** in algorithm design and analysis such as approximation algorithms, online learning, and quantum algorithms.","* **Discuss ethical considerations** in algorithm design and deployment, including fairness, transparency, and accountability.","* **Analyze real-world case studies** to understand the practical applications of algorithms in diverse domains such as finance, healthcare, and e-commerce.","* **Evaluate algorithmic performance** using benchmarking techniques and empirical analysis on real-world datasets.","* **Collaborate with domain experts** to identify relevant algorithms and tailor them to specific big data analytics tasks.","* **Develop algorithmic solutions** for complex problems by combining multiple techniques and algorithms.","* **Understand the impact of data distribution** on algorithm performance and scalability in distributed computing environments.","* **Investigate techniques for algorithm parallelization** and optimization on multi-core processors and GPU accelerators.","* **Discuss emerging trends** in algorithm research such as quantum computing, metaheuristic optimization, and deep reinforcement learning.","* **Explore the application of algorithms** in streaming data processing and real-time analytics for dynamic data environments.","* **Analyze the limitations and challenges** of current algorithms in handling unstructured and semi-structured data types such as text, images, and multimedia.","* **Investigate distributed graph processing frameworks** such as Apache Spark GraphX and Apache Flink Gelly for analyzing large-scale graph datasets.","* **Explore the use of algorithms** in anomaly detection, fraud detection, and cybersecurity applications for identifying patterns and outliers in data streams.","* **Investigate the role of algorithms** in natural language processing (NLP) tasks such as sentiment analysis, named entity recognition, and text summarization.","* **Explore the intersection** of algorithms and computational biology for analyzing genomic data, protein sequences, and biological networks.","* **Analyze the scalability** of algorithms in handling increasingly large datasets and high-dimensional data spaces in big data analytics.","* **Investigate ensemble learning techniques** such as bagging, boosting, and stacking for improving the predictive performance of machine learning models.","* **Explore the application of algorithms** in recommendation systems, personalized marketing, and content recommendation for enhancing user experiences.","* **Analyze the trade-offs** between accuracy, interpretability, and computational complexity in algorithm selection for different analytics tasks.","* **Investigate techniques for algorithmic fairness** and bias mitigation in machine learning models to ensure equitable outcomes across diverse demographic groups.","* **Explore the integration** of algorithms with cloud computing platforms and serverless architectures for scalable and cost-effective data processing.","* **Analyze the impact of algorithmic optimization** on resource utilization, energy consumption, and environmental sustainability in data centers and cloud infrastructures.","AI generated content"],"exams":{"questions":0}},"bda\/6\/2\/b":{"title":"BDA6.2 Data-driven Workflows","id":"bda\/6\/2\/b","requirements":[],"learning outcomes":["* **Identify the components of data-driven workflows**, understanding how data is sourced, processed, and analyzed to inform decisions.","* **Design workflows that integrate data collection, analysis, and interpretation** seamlessly to support continuous improvement and real-time decision making.","* **Implement automation in data workflows** to streamline operations and reduce manual errors, using tools like BPM (Business Process Management) software.","* **Utilize data to predict outcomes and drive decisions**, applying predictive analytics and machine learning models within workflows.","* **Develop metrics and KPIs** to measure the effectiveness of data-driven workflows and guide iterative improvements.","* **Optimize workflow performance** with advanced data strategies, including data blending and real-time data processing.","* **Create robust data governance frameworks** within workflows to ensure data integrity, security, and compliance with regulations.","* **Train teams to leverage data-driven workflows**, emphasizing the importance of data literacy and analytical skills across the organization.","* **Evaluate and select technology platforms** that best support the scalability and complexity of data-driven workflows.","* **Integrate cross-functional data sources** to enrich workflow inputs and enhance the comprehensiveness of data analysis.","* **Develop visualization dashboards** that represent workflow outputs and performance indicators clearly and effectively.","* **Implement feedback mechanisms** in workflows to capture insights and refine processes continuously.","* **Explore case studies of successful data-driven workflows** in various industries to understand best practices and common pitfalls.","* **Address ethical considerations** in automating and managing data-driven decisions, particularly regarding data privacy and bias.","* **Foster a culture of data-driven decision making** within organizations, encouraging proactive data utilization and analysis.","* **Navigate the challenges of integrating legacy systems** with modern data workflow solutions, ensuring seamless data flows across different technologies.","* **Critically assess the impact of data-driven workflows** on organizational efficiency and market responsiveness.","* **Explore emerging technologies** like AI and IoT, and their roles in enhancing data-driven workflows.","* **Conduct workshops and training sessions** to develop hands-on expertise in managing and optimizing data-driven workflows.","* **Lead strategic initiatives** to implement and scale data-driven workflows across large organizations.","AI generated content"],"exams":{"questions":0}},"bda\/6\/b":{"title":"BDA6 Workflows","id":"bda\/6\/b","requirements":[],"learning outcomes":["* Anaylize workflows and investigate the systematic approach.","* Apply data driven workflows for decision making processes.","* Examine the integration of Big data into HPC workflows."],"subskills":["bda\/6\/1\/b","bda\/6\/2\/b","bda\/6\/3\/b"],"exams":{"questions":0}},"bda\/6\/1\/b":{"title":"BDA6.1 Analysis Workflow","id":"bda\/6\/1\/b","requirements":[],"learning outcomes":["* **Define and outline the stages** of a typical big data analysis workflow, from data collection to data interpretation.","* **Develop data ingestion strategies** to effectively gather and store data from various sources, ensuring quality and accessibility.","* **Implement data cleaning and preprocessing techniques** to prepare raw data for analysis, enhancing data quality and usefulness.","* **Utilize exploratory data analysis (EDA)** techniques to summarize characteristics of data and discover initial patterns.","* **Construct models and hypotheses** based on statistical foundations and business intelligence insights.","* **Apply advanced analytical methods** to interpret complex datasets, employing techniques such as machine learning, regression analysis, and clustering.","* **Optimize workflows for efficiency and scalability**, adjusting processes to handle large volumes of data effectively.","* **Automate routine data analysis tasks** using scripting and batch processing to reduce manual effort and increase reproducibility.","* **Validate and refine analytical models** through iterative testing and tuning to improve accuracy and relevance.","* **Communicate results effectively** to stakeholders using visualization tools and presentation techniques.","* **Develop documentation and reporting standards** for analysis workflows to ensure consistency and clarity in outputs.","* **Navigate ethical and compliance issues** related to data analysis, focusing on data privacy, security, and regulatory standards.","* **Integrate new technologies and methodologies** into existing workflows to stay current with industry trends and enhance capabilities.","* **Evaluate the impact of analysis workflows** on business outcomes, demonstrating the value of data-driven decision making.","* **Collaborate in multidisciplinary teams** to bring diverse expertise into the workflow, enhancing the depth and breadth of analytical insights.","* **Critically assess the limitations and biases** in analytical models and workflows, aiming for transparency and objectivity in conclusions.","* **Manage and optimize the use of analytical tools and platforms** within the workflow, including selection and configuration of software and hardware resources.","* **Develop skills in data simulation and synthetic data generation** to test models when actual data is incomplete or unavailable.","* **Implement continuous improvement practices** in analysis workflows to adapt and evolve with organizational needs and technological advances.","* **Lead and manage big data projects** with a focus on strategic planning and cross-functional coordination.","AI generated content"],"exams":{"questions":0}},"bda\/6\/3\/b":{"title":"BDA6.3 Integrating BDA with HPC Workflows","id":"bda\/6\/3\/b","requirements":[],"learning outcomes":["* **Understand the fundamental requirements** for integrating BDA into HPC environments, identifying key challenges and solutions.","* **Design integrated workflows** that leverage the computational power of HPC with the analytical capabilities of BDA.","* **Implement scalable data management strategies** within HPC workflows to handle vast amounts of data efficiently.","* **Utilize parallel computing techniques** in BDA processes to enhance data processing and analysis speeds.","* **Optimize data storage and retrieval** to support intensive BDA tasks within HPC environments.","* **Develop robust data pipelines** that ensure data integrity and availability across distributed computing resources.","* **Apply machine learning and AI algorithms** effectively in HPC settings to solve complex analytical problems.","* **Monitor and manage the performance** of integrated BDA-HPC systems, using tools and practices that enhance reliability and scalability.","* **Address security and privacy issues** relevant to BDA-HPC integrations, implementing comprehensive data protection measures.","* **Evaluate the impact of integrated solutions** on organizational capabilities and business outcomes.","* **Train technical teams** on the effective use of integrated BDA-HPC workflows, focusing on skills development and best practices.","* **Innovate new approaches** for merging BDA with HPC, exploring cutting-edge technologies and methodologies.","* **Navigate regulatory and compliance challenges** associated with deploying BDA-HPC integrations in different industries.","* **Lead cross-functional projects** that utilize integrated BDA-HPC workflows to drive innovation and efficiency.","* **Assess and enhance fault tolerance** and disaster recovery strategies in integrated systems.","* **Explore case studies** where BDA-HPC integration has transformed business processes and analytical capabilities.","* **Engage with industry experts** and thought leaders to stay updated on the latest trends and advancements in BDA-HPC integration.","* **Critically evaluate software and hardware choices** for building effective BDA-HPC environments.","* **Develop strategies for real-time analytics** within HPC workflows, leveraging fast data processing and analysis.","* **Lead strategic planning sessions** to align BDA-HPC integration with broader organizational goals.","AI generated content"],"exams":{"questions":0}},"bda\/3\/2\/b":{"title":"BDA3.2 Object Storage for Big Data","id":"bda\/3\/2\/b","requirements":["* External: Familiarity with data formats and file access in distributed systems","* Internal: None"],"learning outcomes":["* Explain the concept of object storage and how it differs from file\/block storage.","* Identify popular object storage platforms (e.g., S3, Ceph, MinIO) and their use cases in big data.","* Describe data access mechanisms including REST APIs, SDKs, and CLI tools.","* Evaluate performance considerations such as consistency, latency, and throughput.","* Integrate object storage into HPC data pipelines using connectors or abstraction layers.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/3\/b":{"title":"BDA3 Technology","id":"bda\/3\/b","learning outcomes":["* Describe the characteristics and trade-offs of file system technologies used in big data HPC environments.","* Explain how object storage solutions are structured and integrated with HPC and analytics workflows."],"subskills":["bda\/3\/1\/b","bda\/3\/2\/b"],"exams":{"questions":0}},"bda\/3\/1\/b":{"title":"BDA3.1 Parallel File Systems","id":"bda\/3\/1\/b","requirements":["* External: Basic knowledge of file systems and parallel computing","* Internal: None"],"learning outcomes":["* Describe the architecture and operational principles of parallel file systems.","* Compare commonly used systems (e.g., Lustre, GPFS, BeeGFS) in terms of scalability and performance.","* Identify key configuration parameters that influence I\/O throughput.","* Apply best practices for managing I\/O contention and metadata performance.","* Monitor and optimize parallel file system performance for big data workflows.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/7\/2\/b":{"title":"BDA7.2 Scalable Model Evaluation","id":"bda\/7\/2\/b","requirements":["* External: Understanding of evaluation metrics and HPC resource usage","* Internal: BDA7.1 Cross-Validation (recommended)"],"learning outcomes":["* Design scalable evaluation pipelines for large datasets and models.","* Benchmark model performance across multiple hardware configurations or datasets.","* Monitor compute, memory, and I\/O usage during model evaluation.","* Compare models using normalized, reproducible metrics and visualizations.","* Ensure fair evaluation through consistent preprocessing, baselining, and version control.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/7\/b":{"title":"BDA7 Evaluation and Benchmarking","id":"bda\/7\/b","learning outcomes":["* Apply statistical evaluation techniques to assess the generalization and reliability of ML models.","* Use cross-validation and related methods to quantify model performance.","* Benchmark AI models at scale with a focus on consistency, fairness, and comparability."],"subskills":["bda\/7\/1\/b","bda\/7\/2\/b"],"exams":{"questions":0}},"bda\/7\/1\/b":{"title":"BDA7.1 Cross-Validation","id":"bda\/7\/1\/b","requirements":["* External: Familiarity with model training and evaluation","* Internal: BDA5.1.2 Evaluation Metrics (recommended)"],"learning outcomes":["* Define cross-validation and explain its role in evaluating model generalization.","* Compare k-fold, stratified, and leave-one-out cross-validation strategies.","* Implement cross-validation efficiently on large datasets in parallel environments.","* Use cross-validation results to guide model selection and hyperparameter tuning.","* Evaluate statistical significance and variance across validation folds.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"bda\/4\/2\/b":{"title":"BDA4.2 Pre-processing","id":"bda\/4\/2\/b","requirements":[],"learning outcomes":["* **Identify and rectify** inconsistencies, missing values, and outliers in big datasets to enhance data accuracy.","* **Implement normalization** and standardization techniques to ensure data uniformity, making it easier to compare and analyze.","* **Execute transformation methods** such as scaling, encoding, and discretization to tailor data for specific analytical models.","* **Utilize dimensionality reduction** techniques, including PCA, to reduce the number of random variables under consideration, while preserving essential information.","* **Automate repetitive data cleaning tasks** using scripts and libraries to increase efficiency and reduce the likelihood of human error.","* **Develop and apply strategies** for dealing with unstructured data types like text and images, making them amenable to analysis.","* **Assess the impact** of various pre-processing techniques on the quality of datasets and the robustness of subsequent analyses.","* **Employ advanced filtering** to remove redundant or irrelevant data features, focusing analysis on significant attributes.","* **Integrate and reconcile data** from disparate sources to build comprehensive datasets ready for in-depth analysis.","* **Design scalable and reproducible data preprocessing pipelines** that can be applied across various projects and datasets.","* **Optimize preprocessing workflows** for improved performance in high-volume data environments.","* **Explore real-world applications** where effective pre-processing has significantly enhanced the outcomes of data analytics projects.","* **Navigate ethical and legal considerations** in data preprocessing, ensuring compliance with data protection laws and ethical standards.","* **Explore data imputation techniques** to handle missing data effectively, tailoring approaches to the dataset and analysis needs.","* **Implement feature extraction** methods to derive new variables from existing data for enhanced insights and model performance.","* **Use advanced techniques for anomaly detection** to identify significant deviations in data patterns.","* **Develop skills in using automation tools** for data transformation in cloud environments or platforms like Apache Spark.","* **Practice data pre-processing in real-time analytics scenarios**, addressing challenges with streaming data.","* **Learn to preprocess data for specific types of analysis**, such as time series or predictive modeling, customizing methods accordingly.","* **Evaluate the scalability of preprocessing methods** in distributed computing environments, adapting to technology constraints.","* **Address data quality issues systematically**, creating frameworks for assessing and improving data quality.","AI generated content"],"exams":{"questions":0}},"bda\/4\/b":{"title":"BDA4 Data Handling","id":"bda\/4\/b","learning outcomes":["* Examine data preparation for big data workflows.","* Examine data pre-processing for big data workflows.","* Identify visualization techniques for big data.","* Arrange an analysis of a processed data."],"subskills":["bda\/4\/1\/b","bda\/4\/2\/b","bda\/4\/3\/b","bda\/4\/4\/b"],"exams":{"questions":0}},"bda\/4\/1\/b":{"title":"BDA4.1 Preparation","id":"bda\/4\/1\/b","requirements":[],"learning outcomes":["* **Understand the importance** of data preparation in the big data analytics process, including its impact on the quality and reliability of analytical results.","* **Explore techniques** for data acquisition and ingestion from various sources, including databases, file systems, APIs, and streaming platforms.","* **Analyze methods** for data cleaning and preprocessing to address issues such as missing values, outliers, duplicates, and inconsistencies.","* **Understand the principles** of data transformation and normalization to ensure data consistency and compatibility across different formats and structures.","* **Explore the concepts** of feature engineering and extraction for creating new features from raw data to enhance model performance and interpretability.","* **Analyze strategies** for data integration and consolidation to combine disparate datasets into a unified schema for comprehensive analysis.","* **Understand the principles** of data sampling and stratification for creating representative subsets of large datasets to facilitate exploratory analysis and model training.","* **Explore techniques** for handling imbalanced datasets to address challenges associated with unequal class distributions in classification tasks.","* **Analyze methods** for data partitioning and splitting to separate datasets into training, validation, and test sets for model development and evaluation.","* **Understand the concepts** of data anonymization and pseudonymization to protect sensitive information and ensure compliance with privacy regulations.","* **Explore techniques** for handling temporal and spatial data to capture temporal dependencies and spatial relationships in analytical models.","* **Analyze the principles** of data quality assessment and validation to measure the accuracy, completeness, and consistency of datasets.","* **Understand the concepts** of metadata management and documentation to catalog and annotate datasets for traceability and reproducibility.","* **Explore techniques** for data enrichment and augmentation to supplement existing datasets with additional information from external sources.","* **Analyze the role** of data profiling and exploratory data analysis (EDA) in gaining insights into dataset characteristics and identifying patterns and trends.","* **Understand the principles** of data versioning and lineage tracking to manage changes and lineage information throughout the data lifecycle.","* **Explore techniques** for data compression and storage optimization to reduce storage costs and improve data accessibility and retrieval performance.","* **Analyze the challenges** and best practices associated with scaling data preparation workflows for handling increasingly large and complex datasets.","AI generated content"],"exams":{"questions":0}},"bda\/4\/3\/b":{"title":"BDA4.3 Visualization","id":"bda\/4\/3\/b","requirements":[],"learning outcomes":["* **Understand the principles** of data visualization, including the selection of appropriate chart types, color schemes, and data encoding techniques.","* **Develop skills** to use advanced visualization tools and software for creating interactive and static visualizations.","* **Apply best practices** in creating dashboards and reports that effectively communicate the results of data analysis.","* **Explore the use of visualization** for big data in various scenarios such as real-time data streams, large datasets, and complex data structures.","* **Design visualizations** that adhere to accessibility and usability standards to ensure that they are understandable by a diverse audience.","* **Integrate visual analytics** techniques into the visualization process to enable deeper exploration and discovery of data insights.","* **Utilize scripting** with libraries like D3.js for custom visualization solutions that go beyond conventional tools.","* **Critically analyze** the impact of different visualization choices on the interpretation of data.","* **Implement interactive elements** in visualizations to enhance user engagement and data exploration.","* **Assess the effectiveness** of visualizations in conveying the desired message and meeting analytical objectives.","* **Explore case studies** where effective visualization has led to significant business or research outcomes.","* **Navigate ethical considerations** in data visualization, focusing on the responsible representation of information.","* **Leverage advanced visualization techniques** such as heatmaps, geospatial maps, and treemaps.","* **Incorporate time-series analysis** in visualizations to effectively showcase data changes over time.","* **Utilize machine learning algorithms** to identify patterns and outliers, enhancing the insights from visualizations.","* **Implement dynamic and real-time visualizations** for streaming data.","* **Develop custom visual solutions** using programming to extend beyond traditional tools.","* **Evaluate the scalability of visualization tools** to handle Big Data effectively.","* **Integrate visual data discovery tools** for interactive data exploration.","* **Understand visual perception implications** in visualization design.","* **Apply advanced statistical methods** for effective data aggregation in visualizations.","* **Explore the role of AI and automation** in visual data analysis.","AI generated content"],"exams":{"questions":0}},"bda\/4\/4\/b":{"title":"BDA4.4 Analysis","id":"bda\/4\/4\/b","requirements":[],"learning outcomes":["* **Apply statistical methods** to analyze large datasets, interpreting results to make data-driven decisions.","* **Utilize machine learning models** to perform predictive analytics, classifying and forecasting data trends.","* **Implement unsupervised learning techniques** like clustering and dimensionality reduction to discover patterns and relationships in data.","* **Evaluate model performance** using metrics specific to different types of analysis (classification, regression, clustering).","* **Integrate advanced analytics techniques** into business processes to enhance operational efficiency and strategic planning.","* **Develop scripts and algorithms** for automated data analysis, optimizing them for speed and accuracy.","* **Conduct time-series analysis** to forecast future events based on historical data.","* **Use text analytics and natural language processing (NLP)** to extract meaningful information from unstructured data.","* **Perform sentiment analysis** to gauge consumer attitudes and market trends from social media and other textual data.","* **Apply geospatial analysis** to interpret data related to geographic locations and environments.","* **Explore the use of big data analytics in various industry sectors** such as healthcare, finance, retail, and telecommunications.","* **Navigate ethical and legal considerations** in data analysis, ensuring the privacy and security of sensitive information.","* **Participate in hands-on labs and simulations** to apply analytical techniques to real-world datasets.","* **Critically assess the limitations and biases** of data analysis methods, striving for accuracy and fairness in conclusions.","* **Engage in collaborative projects** to tackle complex analysis tasks, sharing insights and methodologies across diverse teams.","* **Stay abreast of emerging trends and technologies** in data analysis, including AI and IoT analytics.","AI generated content"],"exams":{"questions":0}},"adm\/2\/2\/b":{"title":"ADM2.2 Storage","id":"adm\/2\/2\/b","requirements":[],"learning outcomes":["* Understand file systems and remembering","* Local file systems","* file systems over network","* quota","* snapshots","* Describe common usage of NFS in a cluster","* Providing a home directory to users","* Share a software repository","* Describe the key concepts of NFS","* Software\/hardware components","* Security concept of NFSv3","* Performance bottlenecks and caching behavior","* List relevant mount options: protocol, timelimit, nfs-version","* Deploy NFS infrastructure on a server and a client","* Installing required software packages and services on the server","* Specification of exports","* Installing required software packages and the kernel module on the client","* Updating fstab to mount the remote file system","* Examine NFS deployments","* Understand RAID","* Understand Backups","* Understand Data transfer","* Understand Object Stores","* Understand direct disk access over network and remembering","* iSCSI","* VMe over fabric"],"exams":{"questions":0}},"adm\/2\/b":{"title":"ADM2 Cluster Infrastructure","id":"adm\/2\/b","requirements":[],"learning outcomes":["* Review Linux network configuration including routing, nftables, ntp, DNS and DHPC, failover, and TUN\/TAP Networks","* Describe usage of NFS in a cluster and key concepts including software\/hardware components, security, and how to deploy NFS","* Understand RAID including Backups, data transfers, and object stores","* Understand virtualisation and investigate image management, hardware passthrough, and software such as OpenStack, VMWar, QEMU, and libvirt","* Review cables, breakers and disconnects, distributing loads, UPS and power supplies","* Summarize cooling options such as air, water, oil and investigate heat exchangers"],"subskills":["adm\/2\/1\/b","adm\/2\/2\/b","adm\/2\/3\/b","adm\/2\/4\/b","adm\/2\/5\/b"],"exams":{"questions":0}},"adm\/2\/5\/b":{"title":"ADM2.5 Cooling Systems","id":"adm\/2\/5\/b","requirements":[],"learning outcomes":["* Understand Air cooling","* Understand water cooling","* Understand Oil Cooling","* Understand heat exchange to external"],"exams":{"questions":0}},"adm\/2\/1\/b":{"title":"ADM2.1 Network","id":"adm\/2\/1\/b","requirements":["* Networking (ADM1.2)"],"learning outcomes":["* Understand Linux Network Configuration","* Understand Routing","* Understand nftables","* Understand NTP","* Understand DNS and DHCP","* Understand Failover","* Understand Connecting Physical hardware","* Understand TUN\/TAP Networks"],"exams":{"questions":0}},"adm\/2\/3\/b":{"title":"ADM2.3 Virtualization","id":"adm\/2\/3\/b","requirements":[],"learning outcomes":["* Understand networking for virtualisation","* Understand image management","* Understand hardware passthrough","* Understand QEMU","* Understand VMWare","* Understand libvirt","* Understand OpenStack"],"exams":{"questions":0}},"adm\/2\/4\/b":{"title":"ADM2.4 Power","id":"adm\/2\/4\/b","requirements":["* Electricity and Signals (ADM2.10)"],"learning outcomes":["* Understand checking cables","* Understand Breakers and disconnects","* Understand Distributing Load","* Understand Computer\/switch\/etc. power supplies","* Understand UPS"],"exams":{"questions":0}},"adm\/b":{"title":"ADM Administration","id":"adm\/b","requirements":[],"learning outcomes":["* Review theoretic principles such as hardware components, networking, boot process, monitoring, cluster management, and user management.","* Summarize Linux network configuration, and investigate NFS, virtualisation and cooling options","* Describe cluster management with specific tools and investigate workload managers, different types of images as well as deployment, monitoring and IPMI","* Classify software stacks and review installing of software, containers and software containerisation, as well as user installs","* Identify Snakemake's capabilities for configuring its behaviour across a cluster environment and configure a cluster-wide configuration"],"subskills":["adm\/1\/b","adm\/2\/b","adm\/3\/b","adm\/4\/b","adm\/5\/b"],"exams":{"questions":0}},"adm\/5\/b":{"title":"ADM5 Workflow Management Systems - Snakemake","id":"adm\/5\/b","requirements:":["* [[skill-tree:use:6:3:b]]"],"learning outcomes":["* Identify Snakemake's capabilities for configuring its behaviour across a cluster environment.","* Configure and set-up a cluster-wide YAML-based configuration file for Snakemake that will not interfere with other systems (e.g. CWL or Nextflow) and helps avoiding performance issues such as I\/O contention."],"exams":{"questions":0}},"adm\/1\/2\/b":{"title":"ADM1.2 Networking","id":"adm\/1\/2\/b","requirements":[],"learning outcomes":["* Understand IPv4 and IPv6","* Understand Cabling","* Understand switches and cards","* Understand DHCP","* Understand Fabrics"],"exams":{"questions":0}},"adm\/1\/b":{"title":"ADM1 Theoretic Principles","id":"adm\/1\/b","requirements":["* Using knowledge (USE2)"],"learning outcomes":["* Understand the hardware components of a computer and also investigate virtualisation techniques","* Describe the components required for networking including cables, protocols, and fabrics","* Review the boot process and investigate BIOS options, netboot procedures as well as bootloaders","* Summarize monitoring options including CLI monitoring, files under \/proc, and different architectures","* Generalize cluster management options including SSH for administrator access, user management, and system provisioning","* Understand quota for files and compute, user limits and job managers","* Understand the Unix Permission model, how to verify authenticity, 2FA procedures and advanced fencing","* Practice how to model power consumption and understand fundamental concepts such as cables and batteries"],"subskills":["adm\/1\/1\/b","adm\/1\/2\/b","adm\/1\/3\/b","adm\/1\/4\/b","adm\/1\/5\/b","adm\/1\/6\/b","adm\/1\/7\/b","adm\/1\/8\/b"],"exams":{"questions":0}},"adm\/1\/8\/b":{"title":"ADM1.8 Electricity and Signals","id":"adm\/1\/8\/b","requirements":[],"learning outcomes":["* Remember AC and DC electricity","* Remember Electrical safety","* Understand how to model power consumption","* Understand EMI, Ground loops, isolation and shielding","* Understand Communication","* Understand cables and remembering","* Single ended","* differential","* twisted pair","* coax\/twinax","* fiber","* balancing","* bandwidth","* current limits","* Understanding rechargeable batteries and remebering","* Capacity","* currentl limits","* Charging","* safety","* disposal"],"exams":{"questions":0}},"adm\/1\/5\/b":{"title":"ADM1.5 Cluster Management","id":"adm\/1\/5\/b","requirements":["* USE2.4 Remote access"],"learning outcomes":["* Employ SSH connections for administrator access","* Understanding System provisioning","* Understanding user management"],"exams":{"questions":0}},"adm\/1\/1\/b":{"title":"ADM1.1 Architecture","id":"adm\/1\/1\/b","requirements":[],"learning outcomes":["* Understanding Motherboards","* Understanding RAM","* Understanding PCIe","* Understanding DMMA and RDMA","* Understanding Firmware","* Understanding virtualisation and remembering Hypervisors, Emulation, virtualised hardware, and hardware passthrough","* Understanding Temperature and cooling"],"exams":{"questions":0}},"adm\/1\/6\/b":{"title":"ADM1.6 Resource Management","id":"adm\/1\/6\/b","requirements":[],"learning outcomes":["* Understanding User limits","* Understanding Quota","* Understanding job managers"],"exams":{"questions":0}},"adm\/1\/3\/b":{"title":"ADM1.3 Bootprocess","id":"adm\/1\/3\/b","requirements":[],"learning outcomes":["* Understand the BIOS","* Understand Netboot procedure","* Understand the bootloader","* Understand the Init system\/systemd"],"exams":{"questions":0}},"adm\/1\/7\/b":{"title":"ADM1.7 Security","id":"adm\/1\/7\/b","requirements":[],"learning outcomes":["* Remember best practices","* Understand the Unix Permission Model","* Understand how to verify software authenticity and integrity","* Understand Network segmentationa nd firewalls","* Understand SSH-keys and attack monitoring","* Understand 2FA procesures","* Understand Advanced fencing of sensitive data (GDPR)"],"exams":{"questions":0}},"adm\/1\/4\/b":{"title":"ADM1.4 Monitoring","id":"adm\/1\/4\/b","requirements":[],"learning outcomes":["* Understanding CLI monitoring","* Understanding files stored under \/proc","* Understanding Architectures for Organized monitoring tools","* Understanding flapping"],"exams":{"questions":0}},"adm\/3\/2\/b":{"title":"ADM3.2 Resource Management","id":"adm\/3\/2\/b","requirements":["* [[skill-tree:use:2:b]]"],"learning outcomes":["* Understand SLURM","* Understand SGE","* Understand HTConda"],"exams":{"questions":0}},"adm\/3\/b":{"title":"ADM3 Cluster Management","id":"adm\/3\/b","requirements":[],"learning outcomes":["* Understand management tools such as xCAT and Clustore","* Understand workload manager such as SLURM, SGE, or HTConda","* Understand parallel shell programs such as Clustershell","* Understand Stateful and Stateless images as well as how to manage cluster nodes and deployment software","* Understand User management software","* Understand monitoring option such as Prometheus, Telegraf, Grafana, Icinga","* Understand the IMPI system"],"subskills":["adm\/3\/1\/b","adm\/3\/2\/b","adm\/3\/3\/b","adm\/3\/4\/b","adm\/3\/5\/b","adm\/3\/6\/b","adm\/3\/7\/b"],"exams":{"questions":0}},"adm\/3\/5\/b":{"title":"ADM3.5 User Management","id":"adm\/3\/5\/b","requirements":[],"learning outcomes":["* Understand User management software"],"exams":{"questions":0}},"adm\/3\/1\/b":{"title":"ADM3.1 Management Tools","id":"adm\/3\/1\/b","requirements":[],"learning outcomes":["* Understand xCAT","* Understand Clustore"],"exams":{"questions":0}},"adm\/3\/6\/b":{"title":"ADM3.6 Monitoring","id":"adm\/3\/6\/b","requirements":[],"learning outcomes":["* Understand Prometheus","* Understand Telegraf","* Understand Grafana","* Understand Icinga"],"exams":{"questions":0}},"adm\/3\/3\/b":{"title":"ADM3.3 Parallel Shell","id":"adm\/3\/3\/b","requirements":[],"learning outcomes":["* Understand Clustershell"],"exams":{"questions":0}},"adm\/3\/7\/b":{"title":"ADM3.7 IPMI","id":"adm\/3\/7\/b","requirements":[],"learning outcomes":["* Understand the IMPI system"],"exams":{"questions":0}},"adm\/3\/4\/2\/b":{"title":"ADM3.4.2 Stateless Image","id":"adm\/3\/4\/2\/b","requirements":[],"learning outcomes":["* Able to prepare the head node to deploy stateless node images","* Build bootable stateless node images on the head node using chroot or containers","* Setup network configuration of stateless nodes interfaces (Ethernet and High-Speed)","* Configure and start required services on stateless nodes","* Able to deploy stateless images to cluster nodes"],"maintainer":["* Markus Hilger","* Peter Gross\u00f6hme","* HPC Engineers @ Megware"],"exams":{"questions":0}},"adm\/3\/4\/b":{"title":"ADM3.4 System Provisioning","id":"adm\/3\/4\/b","requirements":[],"learning outcomes":["* Understand Stateful images","* Understand Stateless images","* Understand Manage cluster nodes","* Understand deployment software such as Warewulf, Bright, and Ansible"],"subskills":["adm\/3\/4\/1\/b","adm\/3\/4\/2\/b","adm\/3\/4\/3\/b","adm\/3\/4\/4\/b"],"exams":{"questions":0}},"adm\/3\/4\/1\/b":{"title":"ADM3.4.1 Stateful Image","id":"adm\/3\/4\/1\/b","requirements":[],"learning outcomes":["* Able to prepare the head node to deploy stateful node images","* Customize stateful node images with custom partition layouts and software packages","* Setup network configuration of stateful nodes interfaces (Ethernet and High-Speed)","* Configure and start required services on stateful nodes","* Able to deploy stateful images to cluster nodes"],"maintainer":["* Markus Hilger","* Peter Gross\u00f6hme","* HPC Engineers @ Megware"],"exams":{"questions":0}},"adm\/3\/4\/3\/b":{"title":"ADM3.4.3 Manage Cluster Nodes","id":"adm\/3\/4\/3\/b","requirements":[],"learning outcomes":["* Execute a commands on multiple nodes in parallel using the parallel shell","* Able to update software packages on nodes","* Able to gather logs and debug information from nodes and node service processors","* Control nodes using the node service processor (power on\/off, set boot target)","* Able to use the Serial-over-LAN connection for debugging purposes"],"maintainer":["* Markus Hilger","* Peter Gross\u00f6hme","* HPC Engineers @ Megware"],"exams":{"questions":0}},"adm\/3\/4\/4\/b":{"title":"ADM3.4.4 Deployment Systems","id":"adm\/3\/4\/4\/b","requirements":[],"learning outcomes":["* Understand Warewulf","* Understand Bright","* Understand Ansible"],"exams":{"questions":0}},"adm\/4\/2\/b":{"title":"ADM4.2 Containers","id":"adm\/4\/2\/b","requirements":[],"learning outcomes":["* Understand container","* Understand Software containerisation","* Understand container deployment"],"exams":{"questions":0}},"adm\/4\/b":{"title":"ADM4 Software Stack","id":"adm\/4\/b","requirements":[],"learning outcomes":["* Review how to set up a software stack and remember system package managers, building from source, and a module system","* Understand how to use containers and their deployment, as well as software containerisation","* Recognize how user can manage their own software by using HOME Directory installs, SPACK, and similar environments"],"subskills":["adm\/4\/1\/b","adm\/4\/2\/b","adm\/4\/3\/b"],"exams":{"questions":0}},"adm\/4\/1\/b":{"title":"ADM4.1 Setup","id":"adm\/4\/1\/b","requirements":["* [[skill-tree:use:1:5:1:b]]","* [[skill-tree:use:1:5:2:b]]"],"learning outcomes":["* Understand system package manager and remembering","* repository packages","* mirroring and caching","* system upgrades","* rollback","* Understand building from source","* Understand module system and remembering","* dependencies and conflicts","* environmental modules","* Lmod","* Understand SysV Init Scripts","* Understand Systemd Units for services"],"exams":{"questions":0}},"adm\/4\/3\/b":{"title":"ADM4.3 User environment","id":"adm\/4\/3\/b","requirements":[],"learning outcomes":["* Understand Shell initialization","* Understand HOME Directory installs","* Understand SPACK and reme,ber","* concepts","* system configuration","* compilation"],"exams":{"questions":0}},"sd\/2\/2\/b":{"title":"SD2.2 Debugging","id":"sd\/2\/2\/b","learning outcomes":["* Debug a program using simple techniques such as inserting debugging output statements into the source code, e.g. using printf \u2013 also against the background of potential problems with the ordering of the (stdout) output that may exist in parallel environments like MPI.","* Apply the common concepts and workflows when using a debugger (commands like step into, step over, step out, breakpoints).","* Use sophisticated debuggers such as GDB.","* Use sophisticated debuggers such as DDT and TotalView."],"exams":{"questions":0}},"sd\/2\/b":{"title":"SD2 Programming Best Practices","id":"sd\/2\/b","learning outcomes":["* Analyse the benefits of using an IDE for programming.","* Apply the best practices from software engineering regarding programming and debugging.","* Apply programming best practices in order to develop robust and maintainable programs.","* Distinguish different log levels and apply them to a program.","* Dramatize the handling of exceptions for programs.","* Apply coding standards to a programming project.","* Apply testing procedures to a programming project.","* Demonstrate programming of portable code."],"subskills":["sd\/2\/1\/b","sd\/2\/2\/b","sd\/2\/3\/b","sd\/2\/4\/b","sd\/2\/5\/b","sd\/2\/6\/b","sd\/2\/7\/b","sd\/2\/8\/b"],"exams":{"questions":0}},"sd\/2\/8\/b":{"title":"SD2.8 Portability","id":"sd\/2\/8\/b","learning outcomes":["* Understand portability in terms of linked libraries","* Understand code that can be used on different OS"],"exams":{"questions":0}},"sd\/2\/5\/b":{"title":"SD2.5 Exception Handling","id":"sd\/2\/5\/b","learning outcomes":["* Differentiate among the terms \"mistake\", \"fault\", \"failure\", and \"error\".","* Describe exception handling concepts in general (e.g. Errors vs. Exceptions).","* Articulate why it helps to write software that is robust.","* Use best practices for exception handling.","* Describe how exception handling is supported in a specific programming language, e.g. Fortran, C (e.g. <errno.h>), C++ (i.e. try, catch, throw).","* Apply appropriate exception handling in a specific programming language."],"exams":{"questions":0}},"sd\/2\/1\/b":{"title":"SD2.1 Integrated Development Environments","id":"sd\/2\/1\/b","learning outcomes":["* Configure and use integrated development environments (IDEs) like Eclipse, e.g. to seamlessly perform the typical development cycle with the steps edit, build (compile and link), and test."],"exams":{"questions":0}},"sd\/2\/6\/b":{"title":"SD2.6 Coding Standards","id":"sd\/2\/6\/b","learning outcomes":["* Understand and apply coding standards"],"exams":{"questions":0}},"sd\/2\/3\/b":{"title":"SD2.3 Programming Idioms","id":"sd\/2\/3\/b","learning outcomes":["* Describe programming idioms for a specific programming language, e.g. Fortran, Python, C, C++.","* Recognize where programming idioms are violated and to refactor the code to comply to a specific programming idiom.","* Apply programming idioms for a specific programming language, e.g. Fortran, Python, C, C++."],"exams":{"questions":0}},"sd\/2\/7\/b":{"title":"SD2.7 Testing","id":"sd\/2\/7\/b","learning outcomes":["* Understand testing procedures"],"exams":{"questions":0}},"sd\/2\/4\/b":{"title":"SD2.4 Logging","id":"sd\/2\/4\/b","learning outcomes":["* Describe logging in general like log levels etc. (e.g. ERROR, WARN, INFO, DEBUG, TRACE).","* Different logging formats.","* Select appropriate information that should be logged e.g. timestamp, pid, thread, level, loggername) in order to be able to identify the problem.","* Differentiate between structured logging and text logging.","* Apply logging implementations\/libraries for a specific programming language like Fortran, C, C++.","* Develop, maintain, and document a consistent logging concept for a program.","* Implement a logging concept for a program in a specific programming language, e.g. Fortran, C, C++.","* Recognize logging demands and challenges especially for distributed systems.","* Select the most appropriate log format for the context.","* Apply structured logging and text logging."],"exams":{"questions":0}},"sd\/b":{"title":"SD Software Development","id":"sd\/b","learning outcomes":["* Apply software engineering methods and best practices when developing parallel applications.","* Write modular, reusable code by applying software design principles like loose coupling and information hiding.","* Configure and use integrated development environments (IDEs) to seamlessly perform the typical development cycle with the steps edit, build (compile and link), and test.","* Recognize where (parallel) programming idioms are violated and refactor the code to comply with a specific programming idiom.","* Define and establish coding standards and conventions in a project.","* Apply version and configuration management to the development of (parallel) programs in order to track and control changes in the sources and to establish and maintain consistency of the program or software system throughout its life.","* Appropriately document the entire software system.","* Demonstrate using a workflow management system for a given use case."],"subskills":["sd\/1\/b","sd\/2\/b","sd\/3\/b","sd\/4\/b","sd\/5\/b","sd\/5\/b","sd\/7\/b","sd\/8\/b"],"exams":{"questions":0}},"sd\/8\/2\/b":{"error":"skill is incomplete","id":"sd\/8\/2\/b"},"sd\/8\/b":{"title":"SD8 Workflow Management Systems","id":"sd\/8\/b","requirements":["* [[skill-tree:use:1:3:b]]"],"learning outcomes":["* Develop Snakemake workflows for atomization of processes","* Demonstrate contributing Snakemake workflows to the community","* Breakdown workflow management systems and investigate the structure and purpose."],"subskills":["sd\/8\/1\/b","sd\/8\/2\/b","sd\/8\/3\/b"],"exams":{"questions":0}},"sd\/8\/1\/b":{"title":"SD8.1 Programming Snakemake Workflows","id":"sd\/8\/1\/b","requirements":["* [[skill-tree:use:1:3:b]]","* Python Programming"],"learning outcomes":["* Develop Snakemake workflows by utilising rules, input\/output files, and directives such as shell, script, run, \"wrappers\", and resource definitions.","* Integrate custom Python scripts into Snakemake workflows for dynamic data manipulation and resource parameterization.","* Handle automatic software deployment using Conda, or software provisioning with Apptainer\/Singularity or module files.","* Modularize existing Snakemake workflows into semantic units to improve maintainability and scalability."],"exams":{"questions":0}},"sd\/8\/3\/b":{"title":"SD8.3 Workflow Management Systems principles","id":"sd\/8\/3\/b","learning outcomes":["* Explain the purpose and benefits of a Workflow Management System.","* Use Conda, Apptainer\/Singularity, or module files to manage software environments for workflow execution.","* Develop maintainable workflows by building modular workflow structures.","* Configure advanced resource management by setting up workflow-specific resources such as memory, CPU, or cluster configurations.","* Integrate custom Python scripts for quick data manipulation or dynamic resource parameterization.","* Diagnose and resolve workflow errors by identifying failed jobs, analysing logs, and restarting workflows from failed steps."],"exams":{"questions":0}},"sd\/5\/b":{"title":"SD5 Software Quality","id":"sd\/5\/b","learning outcomes":["* Understand Coding Standards","* Understand Code Quality","* Apply software engineering methods and practices especially in the context of high-performance computing.","* Develop parallel programs and to apply software engineering methods and best practices.","* Assess code quality using different metrics, e.g. length of functions, length of files, lines of code, complexity metrics, code coverage.","* Use static code analysis tools in order to calculate the metrics (e.g. http:\/\/cppcheck.sourceforge.net\/).","* Identify bad code structures (known as bad smells) in order to assess the quality of the code design.","* Understand Refactoring","* Apply common code refactorings in order to improve code quality, such as extract method, extract class, rename class and when it is suitable to apply which refactoring.","* Apply refactoring that are specific to programming languages (e.g. Fortran).","* Understand Code Reviews","* Use a review system like Gerrit to organize the code reviews.","* Document code review results and resulting tasks in an issue tracking system (for example Jira).","* Define checklists for code reviews.","* Conduct code reviews in pairs or in a team.","* Document code review results and resulting tasks in an issue tracking system (for example Jira).","* Demonstrate awareness of technical debt during software development and how to pay technical debt."],"exams":{"questions":0}},"sd\/1\/2\/2\/b":{"title":"SD1.2.2 Shared Memory Systems","id":"sd\/1\/2\/2\/b","learning outcomes":["* Detect race conditions and use synchronization mechanisms to avoid them.","* Estimate the problems that may result from erroneous use of synchronization mechanisms (e.g. deadlocks).","* Assess data dependency situations, i.e. an instruction reading the data written by a preceding instruction in the source code, and anti dependencies, i.e. an instruction having to read data before a succeeding instruction overwrites it, and output dependencies, i.e. instructions writing to the same memory location.","* Use data parallelism, e.g. applying parallel streams of identical instructions to different elements of appropriate data structures such as arrays.","* Apply the concept of functional parallelism, i.e. executing a set of distinct functions possibly using the same data.","* Assess the applicability of parallel language extensions like OpenMP.","* Assess concepts like software pipelining, e.g. to optimize loops by out-of-order execution.","* Assess the applicability of parallel language extensions like CUDA as well as their interoperability (e.g. combining OpenACC and CUDA).","* Assess parallel concepts typically used for shared memory systems, e.g. to exploit temporal locality by data reuse with efficient utilization of the memory hierarchy.","* Assess concepts like software pipelining and vectorization principles.","* Assess the influence of control dependencies by jumps, branches, and function calls, e.g. on pipeline filling.","* Assess the applicability of parallel language extensions like OpenACC, and C++ AMP.","* Understand Single Instruction Multiple Data (SIMD)","* Understand Synchronization","* Understand Memory Hierarchy and Data Reuse","* Understand Software Pipelining","* Understand Dependency Pattern and remember","* Data Dependencies","* Control Dependencies","* Understand Language Extensions and remember","* Processes","* CUDA","* OpenACC","* Threading with OpenMP and Pthreads","* C++ AMP","* Vectorization"],"exams":{"questions":0}},"sd\/1\/2\/b":{"title":"SD1.2 Parallel Programming","id":"sd\/1\/2\/b","learning outcomes":["* Assess the parallel nature of algorithms.","* Develop parallel programs using Shared memory programming.","* Develop parallel programs using Message passing.","* Analyse load balancing algorithms and paradigm.","* Demonstrate OP programming middleware.","* Demonstrate GPU programming using CUDA","* Breakdown the difference of accelerators compared with a CPU"],"subskills":["sd\/1\/2\/1\/b","sd\/1\/2\/2\/b","sd\/1\/2\/3\/b","sd\/1\/2\/4\/b","sd\/1\/2\/5\/b","sd\/1\/2\/6\/b","sd\/1\/2\/7\/b"],"exams":{"questions":0}},"sd\/1\/2\/5\/2\/b":{"title":"SD1.2.5.2 XIOS","id":"sd\/1\/2\/5\/2\/b","learning outcomes":["* Describe the XIOS-XML terminology and structuration.","* Test XIOS on a specific architecture.","* Use one or more processes dedicated exclusively to the I\/O management to obtain:","* Simultaneous writing and computing by an asynchronous call.","* Asynchronous transfer of data from clients to servers.","* Asynchronous data writing by each server.","* Use of parallel file system ability via Netcdf4-HDF5 file format to obtain:","* Simultaneous writing in the same single file by all servers.","* No more post-processing rebuilding of the files."],"exams":{"questions":0}},"sd\/1\/2\/5\/b":{"title":"SD1.2.5 I\/O Programming middleware","id":"sd\/1\/2\/5\/b","learning outcomes":["* Assess general concepts of HPC I\/O systems (e.g. parallel file systems) and how to map the data model to the storage system, e.g. by using appropriate I\/O libraries and middleware architectures.","* Demonstrate the use of NetCDF as a parallel data format with different API.","* Demonstrate the use of XIOS."],"subskills":["sd\/1\/2\/5\/1\/b","sd\/1\/2\/5\/2\/b"],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/2\/b":{"title":"SD1.2.5.1.2 NetCDF C++ API","id":"sd\/1\/2\/5\/1\/2\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/b":{"title":"SD1.2.5.1 NetCDF","id":"sd\/1\/2\/5\/1\/b","learning outcomes":["* Examine different NetCDF programming APIs:","* C","* C++","* Java","* Fortra-90","* Python","* Matlab","* IDL","* NCL","* R","* Ruby","* Perl","* Remote","* Examine the NetCDF remote data access","* Analyze NetCDF principle and investigate NetCDF files"],"subskill":["* [[skill-tree:sd:1:2:5:1:1:b]]","* [[skill-tree:sd:1:2:5:1:2:b]]","* [[skill-tree:sd:1:2:5:1:3:b]]","* [[skill-tree:sd:1:2:5:1:4:b]]","* [[skill-tree:sd:1:2:5:1:5:b]]","* [[skill-tree:sd:1:2:5:1:6:b]]","* [[skill-tree:sd:1:2:5:1:7:b]]","* [[skill-tree:sd:1:2:5:1:8:b]]","* [[skill-tree:sd:1:2:5:1:9:b]]","* [[skill-tree:sd:1:2:5:1:10:b]]","* [[skill-tree:sd:1:2:5:1:11:b]]","* [[skill-tree:sd:1:2:5:1:12:b]]","* [[skill-tree:sd:1:2:5:1:13:b]]"],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/8\/b":{"title":"SD1.2.5.1.8 NetCDF NCL API","id":"sd\/1\/2\/5\/1\/8\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/11\/b":{"title":"SD1.2.5.1.11 NetCDF Perl API","id":"sd\/1\/2\/5\/1\/11\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/5\/b":{"title":"SD1.2.5.1.5 NetCDF Python API","id":"sd\/1\/2\/5\/1\/5\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/1\/b":{"title":"SD1.2.5.1.1 NetCDF C API","id":"sd\/1\/2\/5\/1\/1\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file.","* Employ ncgen to generate the C code needed to create the corresponding NetCDF file from a CDL text file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/9\/b":{"title":"SD1.2.5.1.9 NetCDF R API","id":"sd\/1\/2\/5\/1\/9\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/13\/b":{"title":"SD1.2.5.1.13 NetCDF principles","id":"sd\/1\/2\/5\/1\/13\/b","learning outcomes":["* Comply with the best practices for writing NetCDF files.","* Identify protocols, servers, and clients for remote data access through NetCDF interfaces.","* Discuss how features of the enhanced NetCDF-4 format can be applied:","* Groups.","* Multiple unlimited dimensions.","* User-defined types.","* Data compression.","* Create short programs in C that use the NetCDF API to read and write files in the NetCDF classical data model for a given NetCDF data model.","* Create short programs that use the netcdf4-python module to create and read files in NetCDF-4 format for a given NetCDF data model."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/6\/b":{"title":"SD1.2.5.1.6 NetCDF Matlab API","id":"sd\/1\/2\/5\/1\/6\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/3\/b":{"title":"SD1.2.5.1.3 NetCDF Java API","id":"sd\/1\/2\/5\/1\/3\/b","learning outcomes":["* Employ the CDM\/NetCDF-Java to read datasets in various formats.","* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/7\/b":{"error":"skill is incomplete","id":"sd\/1\/2\/5\/1\/7\/b"},"sd\/1\/2\/5\/1\/12\/b":{"title":"SD1.2.5.1.12 NetCDF Remote Data Access","id":"sd\/1\/2\/5\/1\/12\/b","learning outcomes":["* From the user's perspective, one should be able to retrieve data and:","* Obtain information regarding the dataset, without the need to download any files:","* Temporal spatial ranges, available variables, contact info, dataset details.","* Retrieve user-relevant subset of the data (temporal, spatial, and variable subsetting).","* Get data remotely in a variety of ways, including:","* OPeNDAP (DAP2\/DAP4) protocol.","* ncWMS implementing the Web Map Service.","* Web Coverage Service.","* ncISO.","* NetCDF Subset Services.","* Download one file, even if data span multiple files.","* From the data provider's perspective, one should be able to provide data and:","* Catalog data holdings.","* Aggregate data files.","* Provide a quick view of data.","* Easily add information (metadata) to datasets."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/4\/b":{"title":"SD1.2.5.1.4 NetCDF Fortran-90 API","id":"sd\/1\/2\/5\/1\/4\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/5\/1\/10\/b":{"title":"SD1.2.5.1.10 NetCDF Ruby API","id":"sd\/1\/2\/5\/1\/10\/b","learning outcomes":["* Write a two-dimensional array of sample data and read data from this file.","* Write some variables with units attributes and coordinate dimensions and read data variables and attributes from this file.","* Write some four-dimensional variables using a record dimension and read from the variables from this file."],"exams":{"questions":0}},"sd\/1\/2\/1\/b":{"title":"SD1.2.1 Parallel Algorithms","id":"sd\/1\/2\/1\/b","learning outcomes":["* Contrast that some algorithms are embarrassingly (i.e. trivially) parallelizable while their parallelization will vary from easy to hard in practice.","* Determine the computational complexity of algorithms.","* Understand Parallel Nature of Algorithms","* Understand Computational Complexity"],"exams":{"questions":0}},"sd\/1\/2\/6\/b":{"title":"SD1.2.6 GPU Programming with CUDA C\/C++ Programming Fundamentals","id":"sd\/1\/2\/6\/b","learning outcomes":["* Understand the CUDA thread hierarchy:","* Address individual threads within the kernel grid","* Write scalable codes via grid-striding loops","* Configure and modify kernel launch configurations","* Understand the nature of unified memory:","* Understand the necessity for memory transfers","* Understand the functional principles of unified memory","* Understand the potential performance hazards & ways to mitigate those","* Be able to execute kernels concurrently via streams:","* Use streams to fully utilize the GPU","* Use streams to hide memory transfer times","* Be able to query the status of a GPU & potential CUDA errors that occur during program execution"],"maintainer":["* Markus Velten, ZIH Tools Team @ TU Dresden"],"exams":{"questions":0}},"sd\/1\/2\/3\/b":{"title":"SD1.2.3 Message Passing Systems","id":"sd\/1\/2\/3\/b","learning outcomes":["* Understand Synchronization","* Understand Communication Overview and remember","* Blocking","* Non-Blocking","* Point-to-Point","* Collective","* Overlay Networks"],"exams":{"questions":0}},"sd\/1\/2\/7\/b":{"title":"SD1.2.7 Accelerators","id":"sd\/1\/2\/7\/b","learning outcomes":["* Understand FPGA","* Understand GraphCore","* Understand Neuromorphic Computing"],"exams":{"questions":0}},"sd\/1\/2\/4\/b":{"title":"SD1.2.4 Load Balancing","id":"sd\/1\/2\/4\/b","learning outcomes":["* Apply domain decomposition strategies.","* Apply simple scheduling algorithms like task farming to achieve an appropriate distribution of the workloads across the multiple computing resources of the HPC system.","* Apply domain decomposition strategies.","* Apply more sophisticated approaches e.g. based on tree structures like divide-and-conquer or work-stealing to achieve an appropriate distribution of the workloads across the multiple computing resources of the HPC system."],"exams":{"questions":0}},"sd\/1\/b":{"title":"SD1 Programming Concepts for HPC","id":"sd\/1\/b","learning outcomes":["* Develop programs for HPC.","* Develop parallel programs for shared memory systems as well as for message passing systems.","* Develop efficient algorithms and data structures.","* Assess the efficiency and suitability of algorithms and data structures for the respective application."],"subskills":["sd\/1\/1\/b","sd\/1\/2\/b","sd\/1\/3\/b"],"exams":{"questions":0}},"sd\/1\/1\/b":{"title":"SD1.1 Programming Languages","id":"sd\/1\/1\/b","learning outcomes":["* Understand C\/C++","* Understand Fortran","* Understand HPX","* Understand Chip level programming","* Understand Interoperability","* Understand IDEs","* Understand Development Cycle"],"exams":{"questions":0}},"sd\/1\/3\/b":{"title":"SD1.3 Efficient Algorithms and Data Structures","id":"sd\/1\/3\/b","learning outcomes":["* Assess the efficiency of algorithms and data structures, especially with respect to their suitability for typical (scientific) (parallel) programs, e.g. by the help of popular practice-relevant examples."],"exams":{"questions":0}},"sd\/6\/b":{"title":"SD6 Software Design and Software Architecture","id":"sd\/6\/b","learning outcomes":["* Design and develop programs on a higher level of abstraction by applying software architecture and object-oriented design.","* Understand Requirements Elicitation and Analysis","* Describe the role of requirements elicitation and analysis during software development.","* Identify functional and non-functional requirements of the software to be implemented.","* Capture a common terminology, constraints, and assumptions.","* Evaluate the identified requirements against the background of Software Quality.","* Use the identified requirements to validate them by tests.","* Use a predefined template in order to capture the requirements.","* Understand Object-Oriented Design","* Discuss and use the main concepts of object-orientation (classes, interfaces, polymorphism) to design and implement a program.","* Design patterns for HPC.","* Write modular, reusable code by applying software design principles like Separation of concerns, loose coupling, information hiding, DRY, KISS etc. (following best practices like Clean Code by Robert C. Martin).","* Apply object-oriented design and programming to scientific applications.","* Understand Software Architecture","* Describe the importance and impact of software architecture during software development.","* Analyze software architecture principles.","* Understand Software Architecture","* Design the application as a plugin architecture so that the functionality can be extended more easily.","* Design the software architecture of the system based on software architecture patterns.","* Develop an HPC application according to a reference architecture and adhering to the standards\/guidelines given by the reference architecture.","* Implement component-based software architecture for scientific applications.","* Apply component-based architecture design.","* Detect architecture bad smells.","* Identify sub-optimal design decisions in architecture designs (e.g. sub-optimal communication between components).","* Describe the software architecture of a scientific applications, what the most critical architecture decisions are, what the main structures of the system are, what the interfaces of the systems are, how they are designed and how components communicate with each other.","* Discuss which software architecture patterns are typical for a specific type of scientific application and which aspects (e.g. quality attributes) they address.","* Apply design patterns to HPC, e.g. patterns for coding of parallel algorithms and their mapping to various architectures.","* Reference architectures for scientific applications.","* Identify architectural tactics for performance, scalability, etc.","* Apply software architecture principles during the architecture design.","* Interpret the characteristics and the architectural challenges of data-intensive and compute-intensive software systems and how they can be appropriately addressed."],"exams":{"questions":0}},"sd\/3\/2\/b":{"title":"SD3.2 Issue Tracking and Bug Tracking","id":"sd\/3\/2\/b","learning outcomes":["* Describe concepts of issue\/bug tracking systems and their basic concepts like task, sub-task, new feature, story, release planning, sprint planning in order to structure and organize the development process (e.g. assigning tasks to developers, reporting bugs, writing user stories, managing the stages of an issue (to do, in progress, in review, done) etc.)","* Differentiate issue tracking systems, like Jira or Redmine.","* Apply issue tracking in order to manage tasks, bug reports, and other issues occuring during development and enabling task assignment in the team.","* Apply different issue tracking systems, like Jira or Redmine, to the development project.","* Define a consistent workflow in the development team for issue tracking."],"exams":{"questions":0}},"sd\/3\/b":{"title":"SD3 Software Configuration Management","id":"sd\/3\/b","learning outcomes":["* Describe the purpose and importance of software configuration management, especially in the context of high-performance computing.","* Demonstrate Bug and issue tracking for a software project.","* Apply the basic concepts, terms and processes of SCM and apply steps of SCM in an HPC project.","* Detail terms like Configuration Item, Baseline, SCM Directories, Version, Revision, and Release."],"subskills":["sd\/3\/1\/b","sd\/3\/2\/b","sd\/3\/3\/b","sd\/3\/4\/b"],"exams":{"questions":0}},"sd\/3\/1\/b":{"title":"SD3.1 Version Control","id":"sd\/3\/1\/b","learning outcomes":["* Describe the basics of version control systems, e.g. what is version control.","* Discuss the benefits of using version control for software development especially in a team.","* Describe what is branching and merging.","* Assess the difference between distributed and centralized version control systems.","* Apply basic Git concepts.","* Apply basic SVN concepts.","* Use basic git commands such as git add, git commit, git pull, git push.","* Use SVN commands.","* Resolve merge conflicts.","* Apply a specific workflow, such as Feature Branch Workflow, Gitflow Workflow, Centralized Workflow, Forking Worfklow.","* Apply advanced git concepts and commands.","* Apply Git as a version control system.","* Apply SVN as a version control system.","* Apply advanced git concepts, such as pull requests, branches, tags, submodules etc.","* Differentiate types of workflows, such as Feature Branch Workflow, Gitflow Workflow, Centralized Workflow, Forking Worfklow."],"exams":{"questions":0}},"sd\/3\/3\/b":{"title":"SD3.3 Release Management","id":"sd\/3\/3\/b","learning outcomes":["* Describe the basics of release management and what the benefits are of applying a release management process in the context of high-performance computing \"fault\", \"failure\", and \"error\".","* Discuss the differences among Major Release, Minor Release, Emergency Fix (and potentially other types of releases) and what should be contained in each of them.","* Correlate the tasks and steps of release management.","* Apply the steps of the deployment process of the release version and the required dependencies.","* Comply with the best practices of making releases identifiable via version numbers using appropriate version numbering scheme (e.g. using the version control system).","* Characterize the lifecycle of a release (including states such as stable, unstable).","* Differentiate frameworks of release planning and management, e.g. SCRUM release planning and ITIL.","* Apply frameworks of release planning and management like SCRUM release planning or ITIL.","* Classify releases according to release categories (e.g. major, minor, emergency fix).","* Plan and manage releases of scientific software and to document the release including the release notes.","* Apply best practices to make a release identifiable via version numbers using appropriate version numbering scheme (e.g. using the version control system).","* Find the best release management process for the team (e.g. dependending on team size etc.)."],"exams":{"questions":0}},"sd\/3\/4\/b":{"title":"SD3.4 Deployment Management","id":"sd\/3\/4\/b","learning outcomes":["* Describe the basics of dependency management.","* Discuss why different environments for testing, development, production, and staging are necessary.","* Compare the differences between different deployment environments, and what are their specific requirements.","* Use software building environments like make, Autotools, CMake.","* Correlate continuous integration, delivery, and deployment and the differences between them.","* Analyze the benefits, drawbacks, and tradeoffs of continuous integration, delivery, and deployment.","* Set up the production, testing, and development environments.","* Define and preserve reproducible software stacks to make computational results reproducible, e.g. by applying virtualization environments like VirtualBox, Docker, rkt, or BioContainers or tools for defining scientific workflows like Nextflow, or Singularity.","* Configure an environment for continuous integration, delivery, and deployment using a selected continuous integration tool like Jenkins, Buildbot or Travis-CI with basic processing steps like compiling and automated testing.","* Describe the basics of dependency management for different programming languages.","* Use advanced software building environments like Scons and Waf.","* Discuss the challenges for Portability, e.g. for the source code of programs and job scripts to avoid typical compiler-, linker-, and MPI-issues.","* Use a software build and installation framework like EasyBuild."],"exams":{"questions":0}},"sd\/7\/2\/b":{"title":"SD7.2 Software Architecture and Software Design Documentation","id":"sd\/7\/2\/b","learning outcomes":["* Document the different views of the software architecture according to a specific documentation framework, e.g. 4+1 views, Views and Beyond, architecture decision frameworks (e.g. Taylor, Olaf Zimmermann).","* Apply a modeling language for documenting the design and the architecture, e.g. Unified Modeling Language (UML)."],"exams":{"questions":0}},"sd\/7\/b":{"title":"SD7 Documentation","id":"sd\/7\/b","learning outcomes":["* Experiment with requirements and document them.","* Document the entire software architecture and design appropriately.","* Demonstrate source code documentation tools.","* Provide a documentation for developers (e.g. describing the software architecture, for extending the software etc.), which is required for reproducing a program."],"subskills":["sd\/7\/1\/b","sd\/7\/2\/b","sd\/7\/3\/b","sd\/7\/4\/b"],"exams":{"questions":0}},"sd\/7\/1\/b":{"title":"SD7.1 Requirements Documentation","id":"sd\/7\/1\/b","learning outcomes":["* Describe which information needs to be captured in a **requirements** document","* Apply the IEEE standard for software requirements specification for a structured requirement specification.","* Document requirements using a specified template."],"exams":{"questions":0}},"sd\/7\/3\/b":{"title":"SD7.3 Source Code Documentation","id":"sd\/7\/3\/b","learning outcomes":["* Document source code using documentation generators like doxygen, pydoc, or sphinx.","* Produce a consistent source code documentation according to guidelines and best practices."],"exams":{"questions":0}},"sd\/7\/4\/b":{"title":"SD7.4 Documentation for Reproducibility","id":"sd\/7\/4\/b","learning outcomes":["* Document all necessary information for end-users so that they are able to reproduce the results.","* Document the software stack, build instructions, input data, results etc.","* Use tools for literate programming like activepapers knitr, or jupyter to document all necessary information for end-users so that they are able to reproduce the results especially in the context of concurrency."],"exams":{"questions":0}},"sd\/4\/2\/b":{"title":"SD4.2 Extreme Programming","id":"sd\/4\/2\/b","learning outcomes":["* Discuss the principles of extreme programming and when to apply it.","* Apply the principles in the context of an HPC project."],"exams":{"questions":0}},"sd\/4\/b":{"title":"SD4 Agile Methods","id":"sd\/4\/b","learning outcomes":["* Apply agile test development practices in the context of HPC.","* Demonstrate extreme programming.","* Analyse the  concept of SCRUM for project management."],"subskills":["sd\/4\/1\/b","sd\/4\/2\/b","sd\/4\/3\/b"],"exams":{"questions":0}},"sd\/4\/1\/b":{"title":"SD4.1 Test-driven Development and Agile Testing","id":"sd\/4\/1\/b","learning outcomes":["* Describe the challenges of testing scientific applications.","* Discuss test-driven and test-first concepts and understanding the benefits.","* Characterize what constitutes a test strategy.","* Explain that there are different test types, e.g. given by the test pyramid.","* Apply unit testing in a specific programming language using an appropriate unit testing framework, e.g. pfUnit for Fortran, glib testing framework for C.","* Develop (agile) testing strategies.","* Write different test types for the test pyramid.","* Understand Continous Integration (CI) and remeber","* jenkins","* buildbot","* hugo","* Understand Continuous Delivery \/ Deployment (CD)"],"exams":{"questions":0}},"sd\/4\/3\/b":{"title":"SD4.3 SCRUM","id":"sd\/4\/3\/b","learning outcomes":["* Desribe the concepts of SCRUM, e.g. Sprint, Backlog, Planning, Daily meetings\/Stand up meeting, and project velocity.","* Apply practices of SCRUM."],"exams":{"questions":0}},"k\/2\/2\/b":{"title":"K2.2 Bounds for a Parallel Program","id":"k\/2\/2\/b","learning outcomes":["* Explain how performance bounds of the various components of the HPC system (e.g. CPU, caches, memory) can limit the overall performance of a parallel program.","* Explain how performance bounds of the various components of the HPC system (e.g. network, I\/O) can limit the overall performance of a parallel program.","* Explain how performance bounds of the various components of the HPC system (e.g. network, I\/O) can limit the overall performance of a parallel program.","* Explain how performance bounds of the various components of the HPC system (e.g. CPU, caches, memory) can limit the overall performance of a parallel program."],"exams":{"questions":0}},"k\/2\/b":{"title":"K2 Performance Modeling","id":"k\/2\/b","learning outcomes":["* Describe how the performance of parallel programs may be assessed.","* Describe bounds for the performance of parallel programs.","* Describe different performance characteristics."],"subskills":["k\/2\/1\/b","k\/2\/2\/b","k\/2\/3\/b"],"exams":{"questions":0}},"k\/2\/1\/b":{"title":"K2.1 Performance Frontiers","id":"k\/2\/1\/b","learning outcomes":["*  Comprehend that **FLOPS** (Floating Point Operations per Second) as a key measurement is used in two distinct ways:","* To measure the computing power of a computer.","* To measure the performance of applications.","* Comprehend that there are Pitfalls when measuring FLOPScomputing power of a computer:","* Many scientific applications are memory-bound, not CPU bound so the memory buses will become the bottleneck of the application.","* FLOPS say nothing about the performance of the network or the file system.","* FLOP utilization says nothing about the quality of code.","* Comprehend that **Moore\u2019s law** from 1965, revised in 1975, states (in simple terms) that the complexity of integrated circuits and thus the computing power of CPUs for HPC systems, respectively, doubles approximately every two years:","* In the past that was true.","* However, for some time it has been observed that this increase in performance gain is no longer achieved through improvements of processor technology in a sequential sense:","* CPU clock rates, for instance, have not been increased notably for several years.","* It is rather achieved by using many cores for processing a task in parallel.","* Therefore parallel computing and HPC systems will become increasingly relevant in the future.","* Comprehend that **Speedup** defines the relation between the sequential and parallel runtime of a program.","* Comprehend that **Efficiency** defines the relation between the Speedup of a program and the number of processors used to achieve it.","* Comprehend that a good **Scalability** is achieved, when the efficiency remains high while the number of processors is being increased.","* Comprehend that the parallelization of different problems can have different degrees of difficulty:","* Some problems can be parallelized trivially, e.g. rendering (independent) computer animation images.","* However, there are algorithms having a so-called sequential nature, e.g. some tree-search algorithms that have been notoriously difficult to parallelize.","* Typical problems in the field of scientific computing are somewhere in-between these extremes.","* Comprehend that it is an important aspect to use the best known sequential algorithm for speedup comparisons in order to get fair speedup results:","* Conventional Speedup: use the same version of an algorithm (the same program) to measure runtimes T(sequential) and T(parallel).","* Fair Speedup: use the best known sequential algorithm to measure T(sequential).","* Comprehend that **Amdahl's Law** from 1967, states (in simple terms) that there is an upper limit for the maximum speedup achievable with a parallel program which is determined.","* By its sequential, i.e. non-parallelizable part (e.g. for initialization and I\/O operations).","* Or more generally, by the synchronization (e.g. due to unbalanced load) and communication overheads (e.g. for data exchange).","* Comprehend the general Amdahl formula and that in practice it represents a simplified upper limit for the speedup:","* E.g., if the parallelizable part of a program is 99.9% the speedup is in theory limited to 1000.","* But in practice, even speedups above 100 might hardly be achieved in this case if it is additionally considered that overheads for communication and synchronization will also increase when the number of processes is increased.","*  the roofline model, used to provide performance estimates for parallel programs based on multi-core or accelerator processor architectures, by showing inherent hardware limitations"],"exams":{"questions":0}},"k\/2\/3\/b":{"title":"K2.3 Performance Characteristics","id":"k\/2\/3\/b","learning outcomes":["* Understand Performance characteristics of I\/O","* Understand Performance characteristics of CPU Usage","* Understand Performance characteristics of Memory","* Understand Performance characteristics of Communication"],"exams":{"questions":0}},"k\/b":{"title":"K HPC Knowledge","id":"k\/b","learning outcomes":["* Explain the hardware, software, and operation of HPC systems.","* Construct and judge simple performance models for systems and applications.","* Understand that there are performance frontiers.","* Compare different paradigms for the parallelization of applications.","* Construct and execute an HPC workflow on an HPC system.","* Comprehend job scheduling principles.","* Apply a cost model to compute the costs for running a workflow on an HPC system.","* Discuss data management plans."],"subskills":["k\/1\/b","k\/2\/b","k\/3\/b","k\/4\/b","k\/5\/b","k\/6\/b"],"exams":{"questions":0}},"k\/5\/b":{"title":"K5 Modeling Costs","id":"k\/5\/b","learning outcomes":["* Describe the impact of a cluster nodes type (e.g. CPU type, main memory expansion, or GPU extensions) and of the storage media type (SSD, disk, or e.g. tape for long term archiving (LTA) purposes) on its costs.","* Describe how to assess runtime costs for jobs.","* Discuss how to assess the costs for the infrastructure of data and computing centers as well as their personnel costs.","* Explain economic and business aspects, e.g., break-even considerations, when personnel costs for tuning a parallel program and savings through speedups achieved are compared.","* Understand costs of resources and remember","* cluster node types","* Runtime costs","* Storage systems","* Understand costs of Data\/compute centers","* Understand Economic and bisiness aspects for modelling costs","* Understand Personnel costs of tuning and remeber","* Break-even considerations","* savings through speedups"],"exams":{"questions":0}},"k\/1\/2\/2\/b":{"title":"K1.2.2 Network","id":"k\/1\/2\/2\/b","learning outcomes":["* Understand network demands for HPC systems (e.g. high bandwidth and low latency)","* Understand typical network architectures used for HPC systems, like fast Ethernet (1 or 10 Gbit) or InfiniBand","* Understand Demands","* Understand Topologies","* Understand Interconnects"],"exams":{"questions":0}},"k\/1\/2\/b":{"title":"K1.2 Hardware Architectures","id":"k\/1\/2\/b","learning outcomes":["* Differentiate different processing elements such as CPU, FPGA, GPU, and others.","* Demonstrate networking with different topologies and interconnects."],"subskills":["k\/1\/2\/1\/b","k\/1\/2\/1\/b"],"exams":{"questions":0}},"k\/1\/2\/1\/b":{"title":"K1.2.1 Processing elements","id":"k\/1\/2\/1\/b","learning outcomes":["* Understand elementary processing elements like CPUs, GPUs, many-core architectures","* Understand vector systems, and FPGAs","* Understand the NUMA architecture used for symmetric multiprocessing systems where the memory access time depends on the memory location relative to the processor","* Comprehend that in traditional **CPUs** - although CPU stands for Central Processing Unit - there is no central, i.e. single, processing unit any more because today all CPUs have multiple compute cores which all have the same functionality","* Comprehend that **GPUs** (Graphical Processing Units) or **GPGPUs** (General Purpose Graphical Processing Units) were originally used for image processing and displaying images on screens before people started to utilize the computing power of GPUs for other purposes","* Comprehend that **FPGAs** (Field-Programmable Gate Arrays) are devices that have configurable hardware and configurations are specified by hardware description languages","* Comprehend that **FPGAs** are interesting if one uses them to implement hardware features that are not available in CPUs or GPUs (e.g. low precision arithmetic that needs only a few bits)","* Comprehend that **Vector units** are successors of vector computers (i.e. the first generation of supercomputers) and that they are supposed to provide higher memory bandwidth than CPUs","* Comprehend that at an abstract level the high-speed network connects compute units and main memory which leads to three main parallel computer architectures","* **Shared Memory** where all compute units can directly access the whole main memory","* **Distributed memory** where individual computers are connected with a network","* **NUMA** (Non-Uniform Memory Access) combines properties from shared and distributed memory systems, because at the hardware level a NUMA system resembles a distributed memory","* Comprehend that in general, the effort for programming parallel applications for distributed systems is higher than for shared memory systems","* Understand parallelization techniques at the instruction level of a processing element (e.g. pipelining, SIMD processing)","* Understand advanced instruction sets that improve parallelization (e.g., AVX-512)","* Understand hybrid approaches, e.g. combining CPUs with GPUs or FPGAs","* Understand typical network topologies and architectures used for HPC systems, like fat trees based on switched fabrics using e.g. fast Ethernet (1 or 10 Gbit) or InfiniBand","* Understand special or application-specific hardware (e.g. TPUs)"],"exams":{"questions":0}},"k\/1\/b":{"title":"K1 Supercomputers","id":"k\/1\/b","learning outcomes":["* Discuss various system-, hardware-, and I\/O-architectures used for supercomputers, i.e. computers that led the world in terms of processing capacity, and particularly in the speed of calculations, at the time of their introduction, or share key architectural aspects with these computers","* Discuss different hardware architectures.","* Experiment with IO operations on different storage media and also on parallel file systems.","* Outline typical operation of data and computing centers."],"subskills":["k\/1\/1\/b","k\/1\/2\/b","k\/1\/3\/b","k\/1\/4\/b"],"exams":{"questions":0}},"k\/1\/1\/b":{"title":"K1.1 System Architectures","id":"k\/1\/1\/b","learning outcomes":["* Comprehend there are nodes with several functions","* Comprehend that nodes are connected by a high-performance communication network","* Comprehend that global file systems are available on all nodes of the cluster and are convenient because their files can be accessed directly on all nodes","* Quantitatively, parallel file systems offer higher I\/O performance than classic network file systems","* Qualitatively, they allow several processes to write into the same file","* Comprehend the two purposes of the communication network","* Enabling high-speed data communication for parallel applications running on multiple nodes","* Providing a high-speed connection to the disk systems in the cluster","* Understand Share memory systems","* Understand Distributed systems"],"exams":{"questions":1}},"k\/1\/3\/2\/b":{"title":"K1.3.2 Data Reduction Techniques","id":"k\/1\/3\/2\/b","learning outcomes":["* Understand Data reduction techniques"],"exams":{"questions":0}},"k\/1\/3\/b":{"title":"K1.3 Input\/Output","id":"k\/1\/3\/b","learning outcomes":["* Analyse different IO Layers.","* Demonstrate different data reduction techniques.","* Outline data management with cache coherence or staging.","* Outline access patterns for files and also parallel file access."],"subskills":["k\/1\/3\/1\/b","k\/1\/3\/2\/b","k\/1\/3\/3\/b","k\/1\/3\/4\/b"],"exams":{"questions":0}},"k\/1\/3\/1\/2\/b":{"title":"K1.3.1.2 Storage System","id":"k\/1\/3\/1\/2\/b","learning outcomes":["* Understand Object storage","* Understand Burst Buffer","* Understand Network Storage"],"exams":{"questions":0}},"k\/1\/3\/1\/b":{"title":"K1.3.1 I\/O Layers","id":"k\/1\/3\/1\/b","learning outcomes":["* Discuss different media types.","* Discuss different storage systems.","* Discuss different file systems.","* Demonstrate different middle wares.","* Sketch Parallel IO and demonstrate different API.","* Discuss the POSIX interface for IO"],"subskills":["k\/1\/3\/1\/1\/b","k\/1\/3\/1\/2\/b","k\/1\/3\/1\/3\/b","k\/1\/3\/1\/4\/b","k\/1\/3\/1\/5\/b","k\/1\/3\/1\/6\/b"],"exams":{"questions":0}},"k\/1\/3\/1\/5\/2\/b":{"title":"K1.3.1.5.2 PNetCDF","id":"k\/1\/3\/1\/5\/2\/b","learning outcomes":["* Interpret the PNetCDF files structure:","* Header","* Non-record variables (all dimensions specified)","* Record variables (ones with an unlimited dimension)","* Store and retrieve data in PNetCDF.","* Show basic API use and error checking."],"exams":{"questions":0}},"k\/1\/3\/1\/5\/b":{"title":"K1.3.1.5 Parallel I\/O","id":"k\/1\/3\/1\/5\/b","learning outcomes":["* Use parallel I\/O libraries with MPI-IO.","* Use parallel I\/O libraries with PNetCDF.","* Demonstrate parallel IO principles."],"subskills":["k\/1\/3\/1\/5\/1\/b","k\/1\/3\/1\/5\/2\/b","k\/1\/3\/1\/5\/3\/b"],"exams":{"questions":0}},"k\/1\/3\/1\/5\/1\/b":{"title":"K1.3.1.5.1 MPI-IO","id":"k\/1\/3\/1\/5\/1\/b","learning outcomes":["* Employ MPI derived datatypes for expressing the data layout in the file as well as the partitioning of the file data among the communicator processes.","* Specify high-level information about I\/O to the system rather than low-level system-dependent information.","* Collect statistics on the actual read\/write operations performed to validate the MPI-IO performance."],"exams":{"questions":0}},"k\/1\/3\/1\/5\/3\/b":{"title":"K1.3.1.5 Parallel I\/O principles","id":"k\/1\/3\/1\/5\/3\/b","learning outcomes":["* Use parallel I\/O libraries.","* Assess the implications of parallel I\/O on application efficiency.","* Implement an application that utilizes parallel I\/O to store, retrieve and analyze data.","* Improve the I\/O throughput with a parallel I\/O file system.","* List I\/O system simulators developed for parallel I\/O."],"exams":{"questions":0}},"k\/1\/3\/1\/1\/b":{"title":"K1.3.1.1 Media Types","id":"k\/1\/3\/1\/1\/b","learning outcomes":["* Understand different types of storage medias"],"exams":{"questions":0}},"k\/1\/3\/1\/6\/b":{"title":"K1.3.1.6 POSIX","id":"k\/1\/3\/1\/6\/b","learning outcomes":["* Understand the POSIX interface"],"exams":{"questions":0}},"k\/1\/3\/1\/3\/2\/b":{"title":"K1.3.1.3.2 Network File Systems","id":"k\/1\/3\/1\/3\/2\/b","learning outcomes":["* Comprehend that Network file systems are not designed for (very) high I\/O loads."],"exams":{"questions":0}},"k\/1\/3\/1\/3\/b":{"title":"K1.3.1.3 File Systems","id":"k\/1\/3\/1\/3\/b","learning outcomes":["* Understand Local and pseudo File Systems.","* Understand Parallel File System.","* Understand network file systems."],"subskills":["k\/1\/3\/1\/3\/1\/b","k\/1\/3\/1\/3\/2\/b"],"exams":{"questions":0}},"k\/1\/3\/1\/3\/1\/b":{"title":"K1.3.1.3.1 Parallel File System","id":"k\/1\/3\/1\/3\/1\/b","learning outcomes":["* Comprehend that the parallel or cluster aspect is twofold:","* Firstly, the hardware is parallel itself (the file system is provided by several servers that operate in a coordinated way).","* Secondly, parallel I\/O is enabled, i.e. more than one process can consistently write to the same file at the same time."],"exams":{"questions":0}},"k\/1\/3\/1\/4\/2\/b":{"title":"K1.3.1.4.2 NetCDF\/CF Conventions","id":"k\/1\/3\/1\/4\/2\/b","learning outcomes":["* Describe the relationships among elements of the CF-NetCDF conventions and NetCDF entities.","* Locate data in space-time and as a function of other independent variables (coordinates), to facilitate processing and graphics.","* Evaluate programs in C that use the NetCDF API to read and write files in a metadata-aware manner.","* Judge if given metadata meets the CF standard and provides basic understandability for users.","* Design a data model for NetCDF using the CF conventions."],"exams":{"questions":0}},"k\/1\/3\/1\/4\/b":{"title":"K1.3.1.4 Middleware","id":"k\/1\/3\/1\/4\/b","learning outcomes":["* Understand NetCDF","* Understand NetCDF\/CF Conventions","* Understand HDF5"],"subskills":["k\/1\/3\/1\/4\/1\/b","k\/1\/3\/1\/4\/2\/b","k\/1\/3\/1\/4\/3\/b"],"exams":{"questions":0}},"k\/1\/3\/1\/4\/1\/b":{"title":"K1.3.1.4.1 NetCDF","id":"k\/1\/3\/1\/4\/1\/b","learning outcomes":["* Describe the Classic NetCDF Model characteristics and limitations.","* Describe how a 3D-variable\u2019s data values can be represented in a visualization tool.","* Discuss features of the Enhanced Data Model (NetCDF-4): groups, multiple unlimited dimensions, and new types, including user-defined types.","* Analyze the implications of using NetCDF-4 compression on data.","* Justify when to model data using variables or attributes based on characteristics.","* Design and implement a NetCDF data model in CDL text notation for simple datasets using dimensions, variables, attributes, and coordinate variables.","* Apply the in-built NetCDF utilities to create and analyze NetCDF files:","* Examine the CDL data model and the data of a file using ncdump.","* Generate a NetCDF file from a CDL text file using ncgen.","* Convert a NetCDF file from one binary format to another, optionally changing compression and chunk size settings using nccopy."],"exams":{"questions":0}},"k\/1\/3\/1\/4\/3\/b":{"title":"K1.3.1.4.3 HDF5","id":"k\/1\/3\/1\/4\/3\/b","learning outcomes":["* Understand HDF5"],"exams":{"questions":0}},"k\/1\/3\/3\/b":{"title":"K1.3.3 Data Management","id":"k\/1\/3\/3\/b","learning outcomes":["* Understand Cache coherence","* Understand Information lifecycle management","* Understand Staging","* Understand Quota Policies","* Understand Storage tiers","* Understand Archiving"],"exams":{"questions":0}},"k\/1\/3\/4\/b":{"title":"K1.3.4 Access Pattern","id":"k\/1\/3\/4\/b","learning outcomes":["* Understand access patterns","* Understand how different access patterns influence data read and write operations"],"exams":{"questions":0}},"k\/1\/4\/b":{"title":"K1.4 Operation of an HPC System","id":"k\/1\/4\/b","learning outcomes":["* Understand the typical infrastructure of data and computing centers","* Understand economic and business aspects in infrastructure decisions","* Understand administration aspects of an HPC system","* Understand user support aspects (typically on different levels)"],"exams":{"questions":0}},"k\/6\/b":{"title":"K6 Data Management Plan","id":"k\/6\/b","learning outcomes":["* Understanding the risk of losing (knowledge about) data","* Knowing about RDM concepts","* Knowing techniques and abilities to minimize the risk of data loss","* Able to adequately describe and structure data","* Able to apply metadata schemas and create metadata profiles","* Knowing what to consider when publishing data","* Able to find suitable data repositories"],"maintainer":["* Christian L\u00f6schen @ TU Dresden**"],"exams":{"questions":0}},"k\/3\/2\/b":{"title":"K3.2 Parallelization Overheads","id":"k\/3\/2\/b","learning outcomes":["* Comprehend that **Data communication** is necessary for programs that are parallelized for distributed memory computers (if data communication is not necessary the program is called trivially or embarrassingly parallel).","* Comprehend that **Synchronization** plays an important role with shared memory parallelization.","* Comprehend that there are also other sources of parallel inefficiency like:","* Parts of a program that were not parallelized and still run serially.","* Unbalanced loads.","* Comprehend that there are two hardware effects that can reduce the efficiency of the execution of shared-memory parallel programs:","* **NUMA** can lead to noticeable performance degradation if data locality is bad (i.e. if too much data that a thread need is not in its NUMA domain).","* **False sharing** occurs if threads process data that is in the same data cache line, which can lead to serial execution or even take longer than explicitly serial execution of the affected piece of code.","* Comprehend the overheads caused by redundant computations","* Comprehend the problems of execution speed noise (OS jitter, cache contention, thermal throttling, etc.), and typical trade-offs (e.g. reducing the synchronization overhead by increasing the communication overhead)"],"exams":{"questions":0}},"k\/3\/b":{"title":"K3 Program Parallelization","id":"k\/3\/b","learning outcomes":["* Describe the typical parallelization techniques used at the intra- and inter-node level of cluster systems.","* Discuss the causes of parallelization overheads, which eventually prevent efficient use of an increasing number of processing elements.","* Apply domain decomposition strategies (i.e. splitting a problem into pieces that allow for parallel computation).","* Apply Autoparallelization to a program."],"subskills":["k\/3\/1\/b","k\/3\/2\/b","k\/3\/3\/b","k\/3\/4\/b"],"exams":{"questions":0}},"k\/3\/1\/b":{"title":"K3.1 Level of Parallelization","id":"k\/3\/1\/b","learning outcomes":["* Describe parallelization techniques at the intra-node level (e.g. based on basic OpenMP features).","* Apply the message-passing paradigm based on environments like MPI, which is the de-facto standard at the inter-node level for parallelizing programs using more than a single node.","* Understand Intra- and Inter-Node, as well as Multi Level approaches"],"exams":{"questions":0}},"k\/3\/3\/b":{"title":"K3.3 Domain Decomposition","id":"k\/3\/3\/b","learning outcomes":["* Describe typical decomposition strategies to split a domain into subdomains to make it suited for parallel processing.","* Discuss measures like surface to volume ratio.","* Comprehend that in a domain decomposition a region is decomposed, e.g. box is split into smaller ones and a mesh is decomposed into smaller parts, in order to assign these subdomains to processes.","* Comprehend that in order to update a variable that is defined on a site of a mesh, or for a particle, in general data from neighbouring sites or particles is needed.","* Comprehend that some neighbour regions expand beyond the sub-domain of their process, i.e. neighbour regions are partly stored on remote processes (Halo regions) and must be made available on the local process before updating can begin.","* Comprehend that the halo exchange needed in parallel computer simulations is one kind of parallel overhead and has a performance impact, that everybody, who is running such simulations, should know about:","* Relative overhead is approximately proportional to the surface (Halo exchange) to volume (amount of work) ratio of a sub-domain."],"exams":{"questions":0}},"k\/3\/4\/b":{"title":"K3.4 Autoparallelization","id":"k\/3\/4\/b","learning outcomes":["* Describe the auto parallelization capabilities of current compilers (e.g. to automatically parallelize suitable loops), which are applicable at the intra-node level."],"exams":{"questions":0}},"k\/4\/2\/b":{"title":"K4.2 SLURM Workload Manager","id":"k\/4\/2\/b","learning outcomes":["* Run interactive jobs with salloc, a batch job with sbatch.","* Explain the architecture of SLURM, i.e., the role of slurmd, srun and the injection of environment variables.","* Explain the function of the tools: sacct, sbatch, salloc, srun, scancel, squeue, sinfo.","* Explain time limits and the benefit of a backfill scheduler.","* Comprehend that environment variables are set when running a job.","* Comprehend and describe the expected behavior of simple job scripts.","* Comprehend how variables are prioritized when using the command line and a script.","* Change a provided job template and embed them into shell scripts to run a variety of parallel applications.","* Analyze the output generated from submitting to the job scheduler and typically generated errors."],"exams":{"questions":0}},"k\/4\/b":{"title":"K4 Job Scheduling","id":"k\/4\/b","learning outcomes":["* Comprehend the princimples of Job scheduling and why program managing the jobs are required.","* Demonstrate the SLURM workload manager.","* Discuss different Scheduling strategies."],"subskills":["k\/4\/1\/b","k\/4\/2\/b","k\/4\/3\/b"],"exams":{"questions":0}},"k\/4\/1\/b":{"title":"K4.1 Basic principles of Job Scheduling","id":"k\/4\/1\/b","learning outcomes":["* Comprehend the exclusive and shared usage model in HPC.","* Differentiate batch and interactive job submission.","* Comprehend the generic concepts and architecture of resource manager, scheduler, job and job script.","* Explain environment variables as a means to communicate.","* Comprehend accounting principles.","* Explain the generic steps to run and monitor a single job.","* Comprehend scheduling principles (first come first served, shortest job first, backfilling) to achieve objectives like minimizing the averaged elapsed program runtimes, and maximizing the utilization of the available HPC resources.","* Comprehend the differences between **Batch Systems** and **Time-Sharing Systems**.","* Explain the concepts and procedures for resource allocation and job execution in an HPC environment.","* Run interactive jobs and batch jobs.","* Comprehend and describe the expected behavior of job scripts.","* Change provided job scripts and embed them into shell scripts to run a variety of parallel applications.","* Analyze the output generated from a job scheduler and describe the cause of typically generated errors.","* Comprehend accounting principles (billing for the jobs).","* Comprehend the set of terms for performance criteria like:","* Resource Utilization.","* Throughput.","* Waiting Time.","* Execution Time.","* Turnaround Time.","* Comprehend scheduling strategies that increase productivity.","* Comprehend that typical goals of job scheduling are:","* Maximization of resource utilization.","* Maximization of throughput.","* Minimization of waiting time.","* Minimization of turnaround time.","* Comprehend that there is a variety of scheduling algorithms from rather simple to more complex like:","* First-Come-First-Served (FCFS).","* Shortest-Job-First (SJF).","* Priority-based.","* Fair-Share.","* Backfilling.","* Apply advanced scheduling principles (e.g. backfilling) to achieve objectives like minimizing the averaged elapsed program runtimes, and maximizing the utilization of the available HPC resources.","* Discuss sophisticated scheduling principles (e.g. fair share) to achieve objectives like treating the users fair, and maximizing the utilization of the available HPC resources."],"exams":{"questions":0}},"k\/4\/3\/b":{"title":"K4.3 Scheduling strategies","id":"k\/4\/3\/b","learning outcomes":["* Undertand Backfilling","* Undertand Fair Share","* Undertand Shortest Job First","* Undertand First Come First Served"],"exams":{"questions":0}},"ai\/2\/2\/b":{"title":"AI2.2 Scaling and Inference Optimization","id":"ai\/2\/2\/b","requirements":["* External: Basic understanding of AI inference and performance bottlenecks","* Internal: None"],"learning outcomes":["* Describe methods to parallelize inference across multiple compute nodes or GPUs.","* Identify and apply model optimization techniques such as pruning, quantization, and distillation.","* Explain how to balance latency, throughput, and resource usage in production environments.","* Evaluate the impact of batch size, I\/O overhead, and memory footprint on inference performance.","* Use profiling tools to locate bottlenecks and improve inference efficiency in HPC workflows.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/2\/b":{"title":"AI2 Engineering and Infrastructure","id":"ai\/2\/b","learning outcomes":["* Describe the requirements and challenges of hosting AI models in HPC or hybrid infrastructure.","* Identify methods to optimize inference workloads through model and system-level engineering.","* Explain the principles of resource-aware AI deployment, including compute, memory, and energy considerations."],"subskills":["ai\/2\/1\/b","ai\/2\/2\/b","ai\/2\/3\/b"],"exams":{"questions":0}},"ai\/2\/1\/b":{"title":"AI2.1 Hosting AI Models","id":"ai\/2\/1\/b","requirements":["* External: Familiarity with model training and inference concepts","* Internal: None"],"learning outcomes":["* Compare different approaches for deploying AI models in HPC, cloud, and hybrid environments.","* Describe the role of containers (e.g., Docker, Singularity) in hosting AI models.","* Identify tools and frameworks used for serving models (e.g., TorchServe, Triton Inference Server).","* Explain compatibility issues and solutions for model execution on HPC systems.","* Demonstrate how to expose AI models as services or endpoints for internal or external access.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/2\/3\/b":{"title":"AI2.3 Resource-Aware Deployment","id":"ai\/2\/3\/b","requirements":["* External: Familiarity with AI workload characteristics and HPC job environments","* Internal: None"],"learning outcomes":["* Define what makes a deployment \u201cresource-aware\u201d in the context of HPC.","* Select appropriate compute and memory configurations based on model size and workload type.","* Use resource profiling tools to guide allocation decisions.","* Apply strategies to reduce energy consumption during training and inference.","* Explain the trade-offs between performance, resource use, and scheduling constraints.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/b":{"title":"AI Artificial intelligence","id":"ai\/b","learning outcomes":["* Explain the function and design of intelligent agents in HPC-based AI systems and investigate scalability.","* Summarize techniques for fine-tuning AI models, considering performance and resource constraints.","* Describe the requirements and challenges of hosting AI models in HPC or hybrid infrastructure.","* Explain the principles of resource-aware AI deployment, including compute, memory, and energy considerations.","* Summarize techniques used in different models for processing and deployment.","* Identify key regulatory and legal frameworks relevant to AI usage and data handling and also identify principles of responsible AI use.","* Explain how retrieval-augmented generation improves accuracy and grounding in generative systems.","* Describe how AI services are developed and delivered via APIs."],"subskills":["ai\/1\/b","ai\/2\/b","ai\/3\/b","ai\/4\/b","ai\/5\/b","ai\/6\/b"],"exams":{"questions":0}},"ai\/5\/2\/b":{"title":"AI5.2 Retrieval Augmented Generation","id":"ai\/5\/2\/b","requirements":["* External: Familiarity with LLMs and vector search concepts","* Internal: None"],"learning outcomes":["* Define the RAG architecture and explain how it improves generative model performance.","* Describe the components of a retrieval pipeline, including query formulation, embedding, and indexing.","* Identify vector databases and similarity metrics used in AI retrieval tasks.","* Integrate retrieval results into prompt templates or model input streams.","* Evaluate RAG systems based on latency, accuracy, and grounding quality.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/5\/b":{"title":"AI5 Intelligent Interactions and Retrieval Systems","id":"ai\/5\/b","learning outcomes":["* Describe prompting techniques and their influence on generative model behavior.","* Explain how retrieval-augmented generation improves accuracy and grounding in generative systems.","* Understand the architecture and behavior of agentic interfaces designed for iterative or autonomous workflows."],"subskills":["ai\/5\/1\/b","ai\/5\/2\/b","ai\/5\/3\/b"],"exams":{"questions":0}},"ai\/5\/1\/b":{"title":"AI5.1 Prompt Engineering","id":"ai\/5\/1\/b","requirements":["* External: Basic understanding of language models and inference processes","* Internal: None"],"learning outcomes":["* Define prompt engineering and describe its role in controlling generative model output.","* Identify common prompt types (e.g., zero-shot, few-shot, chain-of-thought) and their use cases.","* Design structured prompts for tasks such as summarization, code generation, and question answering.","* Evaluate prompt effectiveness using criteria such as output quality, consistency, and efficiency.","* Apply prompt optimization techniques for performance on large-scale inference systems.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/5\/3\/b":{"title":"AI5.3 Agentic Interfaces and Collaboration","id":"ai\/5\/3\/b","requirements":["* External: Understanding of basic prompting and LLM capabilities","* Internal: AI5.1 Prompt Engineering (recommended)"],"learning outcomes":["* Define agentic interfaces and describe their role in structured decision-making and automation.","* Differentiate between tool-using, memory-enabled, and collaborative agent types.","* Describe the looped reasoning and execution cycles used by AI agents.","* Identify strategies for agent evaluation, feedback integration, and task adaptation.","* Apply best practices for integrating agents into interactive or multi-agent HPC workflows.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/1\/2\/b":{"title":"AI1.2 AI Workflow Management","id":"ai\/1\/2\/b","requirements":["* External: Familiarity with AI training\/inference steps and command-line environments","* Internal: None"],"learning outcomes":["* Define the components of a typical AI workflow (data preprocessing, training, evaluation, deployment).","* Describe the role of workflow engines (e.g., Snakemake, Nextflow, Airflow) in managing AI pipelines.","* Demonstrate how to schedule and monitor multi-stage AI tasks on HPC resources.","* Apply versioning and reproducibility best practices in AI workflow design.","* Understand error handling, checkpointing, and dependency resolution in distributed AI pipelines.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/1\/b":{"title":"AI1 AI System Design and Deployment","id":"ai\/1\/b","learning outcomes":["* Identify major architectural design choices and their impact on AI scalability in HPC environments.","* Describe how AI workflows are coordinated and executed across distributed computing systems.","* Explain the function and design of intelligent agents in HPC-based AI systems.","* Summarize techniques for fine-tuning AI models, considering performance and resource constraints."],"subskills":["ai\/1\/1\/b","ai\/1\/2\/b","ai\/1\/3\/b","ai\/1\/4\/b"],"exams":{"questions":0}},"ai\/1\/1\/b":{"title":"AI1.1 HPC AI Architectures","id":"ai\/1\/1\/b","requirements":["* External: Basic understanding of parallel computing and AI model training","* Internal: None"],"learning outcomes":["* Identify key components of HPC architectures relevant to AI, including GPUs, TPUs, and memory systems.","* Compare different node-level and system-level configurations for AI workloads.","* Explain the role of interconnects (e.g., NVLink, InfiniBand) in distributed AI performance.","* Recognize the impact of hardware-software co-design in AI system performance.","* Describe how different architectural features affect scalability and throughput of AI training and inference tasks.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/1\/3\/b":{"title":"AI1.3 Agents","id":"ai\/1\/3\/b","requirements":["* External: Basic knowledge of reinforcement learning or AI model behavior","* Internal: None"],"learning outcomes":["* Define what constitutes an AI agent in the context of HPC workloads.","* Differentiate between reactive, deliberative, and hybrid agent architectures.","* Explain use cases for agents in model orchestration, data interaction, and simulation-based learning.","* Describe the lifecycle of an agent from initialization to termination in an HPC pipeline.","* Evaluate agent behavior in terms of autonomy, adaptability, and communication.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/1\/4\/b":{"title":"AI1.4 Fine Tuning","id":"ai\/1\/4\/b","requirements":["* External: Understanding of basic deep learning and model training processes","* Internal: None"],"learning outcomes":["* Explain the purpose and benefits of fine-tuning pre-trained models.","* Identify key hyperparameters and architectural considerations during fine-tuning.","* Apply methods for efficient fine-tuning, including layer freezing and learning rate scheduling.","* Describe how fine-tuning strategies differ for large-scale models on HPC infrastructure.","* Recognize potential pitfalls such as overfitting, catastrophic forgetting, and data leakage.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/6\/2\/b":{"title":"AI6.2 Building AI APIs","id":"ai\/6\/2\/b","requirements":["* External: Experience with Python and web frameworks (e.g., Flask, FastAPI)","* Internal: None"],"learning outcomes":["* Design RESTful APIs for AI model inference, including input\/output schema definition.","* Implement API endpoints to expose model functionality securely and efficiently.","* Containerize API services using tools like Docker or Singularity for deployment.","* Deploy APIs in scalable environments using orchestration tools (e.g., Kubernetes, Slurm).","* Monitor, log, and benchmark API performance in production.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/6\/b":{"title":"AI6 AI Services","id":"ai\/6\/b","learning outcomes":["* Describe how AI services are developed and delivered via APIs.","* Explain how external AI models and APIs are integrated into HPC workflows.","* Identify common AI frameworks and how they support scalable, service-oriented deployment."],"subskills":["ai\/6\/1\/b","ai\/6\/2\/b","ai\/6\/3\/b"],"exams":{"questions":0}},"ai\/6\/1\/b":{"title":"AI6.1 Integrating External AI APIs","id":"ai\/6\/1\/b","requirements":["* External: Familiarity with web APIs and basic programming","* Internal: None"],"learning outcomes":["* Identify common external AI APIs and services (e.g., OpenAI, Hugging Face, Google Vision).","* Demonstrate secure API integration using authentication methods like OAuth or API keys.","* Handle latency, rate limits, and failures in long-running HPC jobs using retry\/backoff strategies.","* Apply methods for data formatting and streaming between HPC systems and external APIs.","* Evaluate trade-offs between in-house models and external service integration.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/6\/3\/b":{"title":"AI6.3 AI Frameworks","id":"ai\/6\/3\/b","requirements":["* External: Understanding of deep learning workflows","* Internal: None"],"learning outcomes":["* Compare major AI frameworks such as TensorFlow, PyTorch, JAX, and ONNX.","* Describe the strengths and limitations of each framework in HPC use cases.","* Identify tools for mixed precision training, distributed computing, and hardware acceleration.","* Demonstrate how to port models between frameworks for deployment or optimization.","* Select appropriate frameworks based on model architecture, team skills, and resource constraints.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/2\/b":{"title":"AI3.2 Image Models","id":"ai\/3\/2\/b","requirements":["* External: Basic understanding of computer vision and convolutional neural networks","* Internal: None"],"learning outcomes":["* Identify common architectures used in image modeling (e.g., ResNet, EfficientNet, Vision Transformers).","* Describe the data pipeline requirements for large-scale image datasets.","* Apply techniques for distributed training of image models in HPC environments.","* Explain GPU\/TPU acceleration strategies for image model training and inference.","* Evaluate performance trade-offs between model size, accuracy, and runtime efficiency.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/b":{"title":"AI3 AI Modalities","id":"ai\/3\/b","learning outcomes":["* Explain the structure and application of large language models and how they scale in HPC environments.","* Describe the training and inference workflows of image-based models and their resource requirements.","* Summarize techniques used in audio and voice model processing and deployment.","* Identify the challenges of video generation models and their compute\/memory implications.","* Understand how multimodal models combine inputs from various domains and the synchronization strategies involved.","* Describe graph neural network architectures and their relevance in scientific and relational data modeling.","* Explain the role of domain-specific scientific models in physics-informed AI and simulation-enhanced learning.","* Apply principles of explainable AI (XAI) to interpret predictions and assess model reliability across modalities."],"subskills":["ai\/3\/1\/b","ai\/3\/2\/b","ai\/3\/3\/b","ai\/3\/4\/b","ai\/3\/5\/b","ai\/3\/6\/b","ai\/3\/7\/b","ai\/3\/8\/b"],"exams":{"questions":0}},"ai\/3\/8\/b":{"title":"AI3.8 Explainable AI (XAI)","id":"ai\/3\/8\/b","requirements":["* External: Understanding of basic AI model structure and outputs","* Internal: None"],"learning outcomes":["* Define explainability and distinguish it from transparency and interpretability.","* Identify common XAI methods (e.g., SHAP, LIME, saliency maps) and their applications.","* Apply interpretability techniques to evaluate model decisions in classification or regression tasks.","* Describe use cases for explainability in scientific research, safety-critical systems, and compliance.","* Evaluate trade-offs between model complexity and interpretability.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/5\/b":{"title":"AI3.5 Multimodal Models","id":"ai\/3\/5\/b","requirements":["* External: Basic understanding of multiple data types (e.g., text, image, audio) and neural networks","* Internal: None"],"learning outcomes":["* Define what constitutes a multimodal model and describe its typical input\/output structures.","* Compare fusion strategies (early, late, and hybrid) used to combine modalities in model architectures.","* Explain challenges in synchronizing and batching multimodal inputs during training.","* Identify common datasets and benchmarks used for evaluating multimodal models.","* Describe how HPC systems handle distributed training and scaling of multimodal networks.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/1\/b":{"title":"AI3.1 Language Models (LLMs)","id":"ai\/3\/1\/b","requirements":["* External: Familiarity with basic deep learning concepts and NLP tasks","* Internal: None"],"learning outcomes":["* Describe the transformer architecture and how it underpins most LLMs.","* Explain tokenization strategies and their impact on model efficiency.","* Identify memory and compute bottlenecks in LLM training and inference.","* Compare distributed training strategies used for scaling LLMs across HPC resources.","* Evaluate the performance and limitations of LLMs on different HPC configurations.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/6\/b":{"title":"AI3.6 Graph Neural Networks","id":"ai\/3\/6\/b","requirements":["* External: Understanding of basic machine learning and graph theory concepts","* Internal: None"],"learning outcomes":["* Explain how graph neural networks represent and process relational data.","* Describe core GNN operations such as message passing and aggregation.","* Identify use cases for GNNs in scientific computing, recommendation systems, and bioinformatics.","* Apply techniques for batching and sampling large graphs in distributed training.","* Evaluate performance and scalability of GNNs in multi-node HPC environments.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/3\/b":{"title":"AI3.3 Audio and Voice Models","id":"ai\/3\/3\/b","requirements":["* External: Familiarity with signal processing concepts and neural networks","* Internal: None"],"learning outcomes":["* Describe preprocessing techniques used to transform raw audio into model-ready formats (e.g., spectrograms, MFCCs).","* Compare model architectures suited for audio and voice tasks (e.g., RNNs, CNNs, transformers).","* Explain challenges in training audio models at scale, such as input length variability and I\/O throughput.","* Evaluate model performance across metrics like accuracy, latency, and noise robustness.","* Demonstrate how to optimize audio inference on HPC systems using batching and compression.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/7\/b":{"title":"AI3.7 Scientific Models","id":"ai\/3\/7\/b","requirements":["* External: Familiarity with scientific computing or domain-specific simulation tasks","* Internal: None"],"learning outcomes":["* Describe how AI models can accelerate or complement scientific simulations.","* Explain the concept and application of physics-informed neural networks (PINNs).","* Identify the challenges of incorporating domain knowledge into AI model design.","* Discuss the integration of scientific models with HPC job scheduling and simulation pipelines.","* Evaluate the accuracy, efficiency, and generalizability of scientific AI models.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/3\/4\/b":{"title":"AI3.4 Video Generation","id":"ai\/3\/4\/b","requirements":["* External: Knowledge of deep learning and image\/video data structures","* Internal: None"],"learning outcomes":["* Explain the structure of video generation models and how they differ from static image models.","* Identify key challenges in modeling temporal dynamics and visual coherence.","* Describe the resource demands of video generation, including GPU memory and disk I\/O.","* Apply strategies to manage large-scale video datasets in distributed environments.","* Evaluate performance and quality trade-offs in video synthesis models.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/4\/2\/b":{"title":"AI4.2 Data Privacy and Compliance","id":"ai\/4\/2\/b","requirements":["* External: Familiarity with AI workflows and basic data management concepts","* Internal: None"],"learning outcomes":["* Identify major privacy regulations relevant to AI (e.g., GDPR, HIPAA) and their implications.","* Describe data anonymization and pseudonymization techniques.","* Apply access controls and audit mechanisms for secure data handling in HPC workflows.","* Explain how compliance requirements influence dataset selection, storage, and retention policies.","* Evaluate tools and frameworks that support compliance monitoring and reporting.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/4\/b":{"title":"AI4 Governance and Compliance","id":"ai\/4\/b","learning outcomes":["* Explain the principles of responsible AI development and deployment.","* Identify key regulatory and legal frameworks relevant to AI usage and data handling.","* Describe how data provenance, auditability, and reproducibility are maintained in HPC AI workflows."],"subskills":["ai\/4\/1\/b","ai\/4\/2\/b","ai\/4\/3\/b"],"exams":{"questions":0}},"ai\/4\/1\/b":{"title":"AI4.1 Responsible AI Use","id":"ai\/4\/1\/b","requirements":["* External: General understanding of AI system design and deployment","* Internal: None"],"learning outcomes":["* Define key principles of responsible AI, including fairness, non-discrimination, and inclusiveness.","* Identify potential sources of bias in AI data and models and methods to mitigate them.","* Describe strategies for ensuring transparency and explainability in model outputs.","* Recognize ethical risks in deploying large-scale AI systems and how to address them.","* Apply responsible AI guidelines or frameworks (e.g., EU AI Act, OECD Principles) to HPC workflows.","** Caution: All text is AI generated **"],"exams":{"questions":0}},"ai\/4\/3\/b":{"title":"AI4.3 Data Provenance and Auditability","id":"ai\/4\/3\/b","learning outcomes":["* Define data provenance and explain its importance in scientific and regulated AI use cases.","* Identify tools and metadata standards used for tracking data lineage.","* Describe how audit trails can be maintained across distributed HPC workflows.","* Implement strategies to ensure reproducibility of AI experiments, including versioning of data and models.","* Evaluate systems that integrate provenance tracking with workflow engines or data lakes.","** Caution: All text is AI generateIntelligent Interactions and Retrieval Systems"],"exams":{"questions":0}},"use\/2\/2\/b":{"title":"USE2.2 Job Scripts","id":"use\/2\/2\/b","requirements":[],"learning outcomes":["* Use the command-line interface.","* Write robust job scripts, e.g. to simplify job submissions by the help of automated job chaining.","* Select the appropriate software environment.","* Use a workload manager to allocate HPC resources for running a parallel program interactively.","* Recognize cost aspects.","* Measure system performance as a basis for benchmarking a parallel program.","* Benchmark a parallel program.","* Tune a parallel program from the outside via runtime options.","* Apply the workflow for tuning."],"exams":{"questions":0}},"use\/2\/b":{"title":"USE2 Running of Parallel Programs","id":"use\/2\/b","learning outcomes":["* Use a workload manager like SLURM or TORQUE to allocate HPC resources (e.g. CPUs) and to submit a batch job.","* Write robust job scripts, e.g. to simplify job submissions by the help of automated job chaining."],"subskills":["use\/2\/1\/b","use\/2\/2\/b"],"exams":{"questions":0}},"use\/2\/1\/b":{"title":"USE2.1 Job Scheduling","id":"use\/2\/1\/b","learning outcomes":["* Run parallel programs in an HPC environment.","* Use the command-line interface.","* Write robust job scripts, e.g. to simplify job submissions by the help of automated job chaining.","* Select the appropriate software environment.","* Use a workload manager like SLURM or TORQUE to allocate HPC resources (e.g. CPUs) and to submit a batch job.","* Consider cost aspects.","* Measure system performance as a basis for benchmarking a parallel program.","* Benchmark a parallel program.","* Tune a parallel program from the outside via runtime options.","* Apply the workflow for tuning."],"exams":{"questions":0}},"use\/b":{"title":"USE Use of the HPC Environment","id":"use\/b","learning outcomes":["* Apply tools provided by the operating system to navigate and manage files and executables.","* Explain what programs and processes are and how the CLI interacts with and manages them.","* Use a workload manager to allocate HPC resources and run job using scripts.","* Select the software environment to build existing OpenSource projects","* Develop novel parallel applications effectively.","* Design and deploy scripts that automatize repetitive tasks.","* Construct workflows that utilize remote (distributed) environments to execute a parallel workflow.","* Illustrate the use of post processing tools for analyzing and visualizing results."],"subskills":["use\/1\/b","use\/2\/b","use\/3\/b","use\/4\/b","use\/5\/b","use\/6\/b","use\/7\/b"],"exams":{"questions":0}},"use\/8\/2\/b":{"title":"USE8.2 AI Infrastructure","id":"use\/8\/2\/b","requirements":[],"learning outcomes":["* Access AI tools via a web interface and understand basic interaction workflows.","* Send requests to AI models using API interfaces and interpret structured responses.","* Understand how to use API keys, endpoints, and basic request formatting (e.g., JSON).","* Distinguish between synchronous and asynchronous inference requests.","* Integrate simple AI responses into command-line or scripting workflows for automation."],"exams":{"questions":0}},"use\/8\/b":{"title":"USE8 AI Services","id":"use\/8\/b","requirements":[],"learning outcomes":["* Understand the purpose and availability of AI services within an HPC environment.","* Identify common methods of interacting with AI models, including prompt-based and API-based usage.","* Describe the difference between web interface access and programmatic access to AI tools.","* Recognize the scope and limitations of using hosted AI tools without managing the underlying infrastructure."],"subskills":["use\/8\/1\/b","use\/8\/2\/b"],"exams":{"questions":0}},"use\/8\/1\/b":{"title":"USE8.1 Prompt Engineering","id":"use\/8\/1\/b","requirements":[],"learning outcomes":["* Understand how the structure and phrasing of a prompt can influence the output of a language model.","* Identify common prompt formats used for AI tasks such as summarization, generation, classification, and translation.","* Recognize and avoid common prompt-related errors (e.g., ambiguous wording, lack of context).","* Refine prompts iteratively to improve output quality.","* Apply techniques such as role prompting, few-shot prompting, and instruction-based prompting."],"exams":{"questions":0}},"use\/5\/b":{"title":"USE5 Automating Common Tasks","id":"use\/5\/b","reqirements":["* [[skill-tree:use:2:b]]"],"learning outcomes":["* Recognize which tasks are eligible for automatization","* Create simple scripts to execute shell commands","* Distinguish different types of commands and loops","* Articulate when it is more efficient to use a shell script rather than a program and when its the opposite"],"exams":{"questions":0}},"use\/1\/2\/b":{"title":"USE1.2 Shell Scripts","id":"use\/1\/2\/b","requirements":["* [[skill-tree:use:1:1:b]]"],"learning outcomes":["* Create a basic shell script that executes a sequence of programs.","* Design a script using the bash construct \"if\" that handles conditional execution based on:","* Performing a test for the existing of a file\/directory\/variable,","* Testing for the presence of certain text in a file,","* Design a script that performs a task repeatedly using the bash \"for\" or \"while\" loop.","* Utilize debugging options for troubleshooting of shell programs:","* Options to bash: -e, -x","* Use shell functions to break large, complex sequences, into a series of small functions.","* Learn how to manipulate filenames.","* Learn to manage temporary files:","* Choose an adequate file system (or top directory) for temporary files.","* Automatically generate a unique folder name for temporary files.","* Automatically delete temporary folder whenever the script exits."],"exams":{"questions":5}},"use\/1\/b":{"title":"USE1 Cluster Operating System","id":"use\/1\/b","learning outcomes":["* Describe the command line interface and the bash shell as well es perform basic commands.","* Use and write basic shell scripts, e.g., to automate the execution of several commands.","* Sketch the organization of the typical file system tree.","* Utilize essential tools to navigate and manage files.","* Understand how to access a Cluster via SSH.","* Understand terminal multiplexers and use them to run longer commands without keeping a connection alive.","* Utilize tools to transfer data between a desktop\/laptop system and a remote HPC system.","* Utilize package management tools that provide access to a wide variety of software.","* Demonstrate the usage of an editor to open a file in write mode and save the file and define why CLI based editors are useful in an HPC environment.","* Utilize screen or tmux to preserve a session between logins and classify usecases for these three operations:","* Creation of a session.","* Detaching from the current session.","* Resuming a previous session.","* Illustrate the use of CTRL-Z and CTRL-C in combination with the commands jobs and fg to send the signals SIGINT and SIGTSTP.","* Experiment with the kill command and send the signals SIGKILL, SIGINT, SIGTSTP, also via the numbers instead of the names to a specific process id"],"subskills":["use\/1\/1\/b","use\/1\/2\/b","use\/1\/3\/b","use\/1\/4\/b","use\/1\/5\/b","use\/1\/6\/b","use\/1\/7\/b","use\/1\/8\/b"],"exams":{"questions":7}},"use\/1\/8\/b":{"title":"USE1.8 Manage background processes and signals","id":"use\/1\/8\/b","requirements":["* [[skill-tree:use:1:1:b]]"],"learning outcomes":["* Understand interrupting or aborting of a program execution using CTRL-C and CTRL-Z.","* Describe the difference between SIGINT and SIGTSTP.","* Demonstrate the use of CTRL-Z in combination with the commands jobs and fg.","* Understand what a Process ID is and how it can be used to identify any programs.","* Demonstrate how the command jobs can display the process id.","* Understand the behaviour of the kill command.","* Demonstrate the kill command and send the signals SIGKILL, SIGINT, SIGTSTP, also via the numbers instead of the names"],"exams":{"questions":0}},"use\/1\/5\/2\/b":{"title":"USE1.5.2 Spack","id":"use\/1\/5\/2\/b","learning outcomes":["* Describe the concept of configurations and used hashing.","* Query information about packages:","* list","* info","* find (also use -d to reveal their dependencies)","* spec","* Load\/Unload software modules.","* Describe the spec syntax to specify package configurations."],"exams":{"questions":0}},"use\/1\/5\/b":{"title":"USE1.5 Software Environment","id":"use\/1\/5\/b","learning outcomes":["* Query available software using the package manager and select the appropriate versions for deployment in the session environment.","* Describe the MODULEPATH variable and understand shell variables relevant for executing and building software","* Query information about packages using SPACK and install as well as load and unload software modules","* Understand what conda environments are for and how a user can install their own environments"],"subskills":["use\/1\/5\/1\/b","use\/1\/5\/2\/b","use\/1\/5\/3\/b"],"exams":{"questions":2}},"use\/1\/5\/1\/b":{"title":"USE1.5.1 Environment Modules","id":"use\/1\/5\/1\/b","requirements":[],"learning outcomes":["* Comprehend that Modules can have dependencies and conflicts:","* A Module can enforce other Modules that it depends on must be loaded before the Module itself can be loaded.","* Conflicting modules must not be loaded at the same time (e.g. two versions of a compiler).","* Query information about packages:","* list","* avail","* whatis","* search","* display","* help","* Load\/Unload software modules:","* load\/unload","* purge","* swap","* switch","* Describe the MODULEPATH variable.","* Describe the general dependency structure of software.","* Describe how a package manager makes software available.","* Understand shell variables relevant for executing and building software:","* PATH for executables","* LD\\_LIBRARY\\_PATH for libraries","* MANPATH for manual pages (man command)","* PKG\\_CONFIG\\_PATH for pkg-config","* Manipulate shell variables to include additional software:","* Setting shell variables for a single program by prefixing or by using export."],"exams":{"questions":0}},"use\/1\/5\/3\/b":{"title":"USE1.5.3 Conda environments","id":"use\/1\/5\/3\/b","requirements":["* [[skill-tree:use:1:5:1:b]]"],"learning outcomes":["* Understand what conda environments are for","* Understand the difference between a module system and conda environment","* Discuss how such an environment can be constructed and what dependencies it has","* Prepare a conda environment for basic python packages","* Analyse the loading time for a conda environment for different file storage location"],"exams":{"questions":0}},"use\/1\/1\/b":{"title":"USE1.1 Command Line Interface","id":"use\/1\/1\/b","requirements":[],"learning outcomes":["* Use the bash shell to execute individual programs with arguments.","* List the set of basic programs and their tasks:","* pwd","* ls","* cd","* whoami","* sleep","* kill","* echo","* clear","* man","* vi, vim, emacs, nano","* exit","* ...","* Demonstrate interrupting or aborting of a program execution using CTRL-C.","* Use the available help of a program (--help argument and the man pages).","* Use the shell history to search and execute previously executed commands.","* Demonstrate setting and printing shell variables.","* Show all currently set variables and set as well as unset one manually.","* Identify potential special characters that must be handled with care.","* List strings that could refer to files\/directories","* Utilize escaping to ensure correct handling of arguments.","* Understand wildcard characters to select a group of files\/directories: \"*\", \"?\", \"[-,]\""],"material":[{"@type":"Book","identifier":"m1","name":"The great MPI course","description":"a description...","creator":[{"name":"Its-a Leader","familyName":"Leader","givenName":"Its-a"}],"contributor":[{"name":"Hans Lastname"}],"inLanguage":"en","teaches":["use\/1\/1\/b","use\/1\/2\/b"],"educationalAlignment":{"educationalFramework":"HPC CF Competence Standard v1.0","targetUrl":"https:\/\/www.hpc-certification.org\/cs\/v1.0"},"keywords":"test,test2","educationalLevel":"beginner","url":"https:\/\/xx","datePublished":"2020-10-03","dateCreated":"2020-10-01"}],"events":[{"identifier":"e1","name":"interesting course","description":"description","inLanguage":"en","location":{"url":"http:\/\/","PostalAddress":{"addressCountry":"","addressRegion":"","postalCode":"","streetAddress":""},"VirtualLocation":{"name":"Zoom","url":"https:\/\/"}},"maximumAttendeeCapacity":30,"url":"https:\/\/xx","datePublished":"2020-10-03","startDate":"2020-11-03T10:20:00","endDate":"2020-11-03T12:20:00"}],"exams":{"questions":4}},"use\/1\/6\/b":{"error":"skill is incomplete","id":"use\/1\/6\/b"},"use\/1\/3\/b":{"title":"USE1.3 UNIX File System Tree","id":"use\/1\/3\/b","requirements":["* [[skill-tree:use:1:1:b]]"],"learning outcomes":["* Describe the organization of a hierarchical file system.","* Explain the basic UNIX permission model and metadata of files.","* Describe the Filesystem Hierarchy Standard and the relevance of the directories:","* etc","* home","* opt","* lib and \/usr\/lib","* bin and \/usr\/bin","* sbin and \/usr\/sbin","* tmp","* Utilize tools to navigate and traverse the file system:","* ls (-R, -l)","* cd","* pushd\/popd","* stat","* Use tools to read files:","* cat","* head\/tail","* less\/more","* cmp","* Utilize tools to manipulate the file system:","* mkdir\/rmdir","* touch","* cp\/mv","* ln","* Utilize tools to identify and manipulate permissions:","* chmod","* chown\/chgrp"],"exams":{"questions":8}},"use\/1\/7\/b":{"title":"USE1.7 Terminal multiplexer","id":"use\/1\/7\/b","requirements":["* [[skill-tree:use:1:1:b]]","* [[skill-tree:use:1:4:b]]"],"learning outcomes":["* Give examples for use cases for such a multiplexer.","* Discuss using such a session for longer data transfers using SSH.","* Review editing files using a terminal multiplexer","* Utilize screen or tmux to preserve a session between logins:","* Creation of a session.","* Detaching from the current session.","* Resuming a previous session."],"exams":{"questions":0}},"use\/1\/4\/b":{"title":"USE1.4 Remote Access","id":"use\/1\/4\/b","requirements":["* [[skill-tree:use:1:1:b]]"],"learning outcomes":["* Describe how SSH-authentication works:","* Password authentication.","* Public-key authentication.","* The role of an authentication agent and the security implications.","* Generate an SSH public\/private key under Linux.","* Register a key for public-key authentication using ssh-copy-id or other local methods.","* Perform a remote login from Linux using SSH.","* Use SSH-agent or equivalents.","* Use Agent forwarding to connect to a third HPC system from an HPC system that you logged into with ssh from your computer.","* Know when to use and how to create a config file.","* Utilize tools to transfer data between the local and remote system:","* scp","* sftp","* rsync (-avz)"],"exams":{"questions":0}},"use\/6\/2\/b":{"error":"skill is incomplete","id":"use\/6\/2\/b"},"use\/6\/b":{"error":"skill is incomplete","id":"use\/6\/b"},"use\/6\/1\/b":{"error":"skill is incomplete","id":"use\/6\/1\/b"},"use\/6\/3\/b":{"error":"skill is incomplete","id":"use\/6\/3\/b"},"use\/6\/4\/b":{"error":"skill is incomplete","id":"use\/6\/4\/b"},"use\/3\/b":{"title":"USE3 Building of Parallel Programs","id":"use\/3\/b","requrements":["* [[skill-tree:use:1:b]]","* [[skill-tree:use:2:b]]"],"learning outcomes":["* Build parallel programs, e.g. via open sources packages.","* Run parallel programs in an HPC environment.","* Use a compiler and to assess the effects of optimization switches available for the relevant compilers (e.g. GNU, Intel).","* Use a linker and to assess the effects of linker specific options and environment variables (e.g. -L and LIBRARY_PATH, -rpath and LD_RUN_PATH).","* Configure the relevant settings (e.g. by setting compiler and linker options), which determine how the application ought to be built with regard to the parallelization technique(s) used (e.g. OpenMP, MPI).","* Use software building environments like make, Autotools, CMake.","* Run parallel programs in an HPC environment.","* Use a compiler and to assess the effects of optimization switches available for compilers commercially available (e.g. PGI, NAG).","* Use efficient open-source libraries (e.g. OpenBLAS, FFTW) or highly optimized vendor libraries (e.g. Intel-MKL, IBM-ESSL).","* Configure the relevant settings (e.g. by setting compiler and linker options), which determine how the application ought to be built with regard to the parallelization technique(s) used (e.g. OpenACC, C++ AMP).","* Use the profile guided optimization (PGO) technique.","* Use software building environments like Scons, Waf, make, Autotools, CMake."],"exams":{"questions":0}},"use\/7\/b":{"title":"USE7 Post-processing Tools","id":"use\/7\/b","learning outcomes":["* Understand types of post processing","* Analyze post processing workflows","* Understand visualization tool such as Paraview","* Understand CDO"],"exams":{"questions":0}},"use\/4\/b":{"error":"skill is incomplete","id":"use\/4\/b"},"pe\/2\/2\/b":{"title":"PE2.2 I\/O Performance","id":"pe\/2\/2\/b","learning outcomes":["* Select performance models to assess and optimize the application I\/O performance.","* Identify tools capable of predicting the behavior of applications in HPC.","* Apply methods to manipulate workloads to improve achievable performance.","* Understand metrics that describe relevant I\/O performance characteristics.","* Select tools to analyze I\/O performance of applications.","* Apply the performance analysis tools to applications and workflows.","* Interpret the results of I\/O analysis tools and distill optimization strategies for the application or system configurations."],"exams":{"questions":0}},"pe\/2\/b":{"title":"PE2 Measuring System Performance","id":"pe\/2\/b","learning outcomes":["* Measure the system performance by the help of standard tools and by profiling in order to assess the runtime behavior of parallel programs.","* Compute the utilization of a machine by various measures.","* Discover the IO performance of a program and select an appropriate model."],"subskills":["pe\/2\/1\/b","pe\/2\/2\/b","pe\/2\/3\/b"],"exams":{"questions":0}},"pe\/2\/1\/b":{"title":"PE2.1 Utilization Principles","id":"pe\/2\/1\/b","requirements":[],"learning outcomes":["* Understand CPU vs. Elapsed Times","* Understand Shared and Unshared Memory","* Understand I\/O Statistics (Devices and File Systems)","* Understand Page Faults"],"exams":{"questions":0}},"pe\/2\/3\/2\/b":{"title":"PE2.3.2 Scalasca","id":"pe\/2\/3\/2\/b","learning outcomes":["* Able to automatically run Score-P instrumented measurements (including automatic","analysis) of applications containing","- MPI, and","- OpenMP and\/or Pthreads.","* Able to identify waiting time in the application measurement","* Able to identify the causing delay (imbalance) causing waiting time","* Derive solution hypotheses from the analysis report to resolve the","performance problems identified."],"maintainer":["* Marc-Andr\u00e9 Hermanns, HPC Group, IT Center, RWTH Aachen University"],"exams":{"questions":0}},"pe\/2\/3\/b":{"title":"PE2.3 Profiling tools","id":"pe\/2\/3\/b","learning outcomes":["* Demonstrate the use of Score-P for collecting program traces.","* Demonstrate the use of Scalasca for analyzing traces.","* Demonstrate the analysis of program traces using Vampir.","* Understand Darshan.","* Demonstrate PIKA to check the performance of anyprogram without instrumenting it.","* Demonstrate collecting traces of a program usig L02s.","* Demonstrate analysis program from NVIDIA for CUDA code."],"subskills":["pe\/2\/3\/1\/b","pe\/2\/3\/2\/b","pe\/2\/3\/3\/b","pe\/2\/3\/4\/b","pe\/2\/3\/5\/b","pe\/2\/3\/6\/b","pe\/2\/3\/7\/b"],"exams":{"questions":0}},"pe\/2\/3\/5\/b":{"error":"skill is incomplete","id":"pe\/2\/3\/5\/b"},"pe\/2\/3\/1\/b":{"title":"PE2.3.1 Score-P","id":"pe\/2\/3\/1\/b","learning outcomes":["Able to instrument applications including one or more parallel paradigms from the following:","* MPI","* SHMEM","* OpenMP","* Pthreads","Able to instrument applications including at least one specialized form of measurement from the following:","* Accelerator usage","* I\/O performance","* Hardware counters","* Memory usage","Able to score measurements and effectively describe whether a measurement includes unacceptable perturbation","Able to create and use filter files to improve measurement quality and relevance","Able to manage measurement data through environment variables and scripting","* Able to automate naming\/filing of experiments in appropriate locations","* Able to read and understand configuration and manifest files generated by Score-P"],"maintainer":["* Bert Wesarg, William (Bill) Williams, ZIH Tools Team @ TU Dresden"],"exams":{"questions":0}},"pe\/2\/3\/6\/b":{"title":"PE2.3.6 Lo2s","id":"pe\/2\/3\/6\/b","learning outcomes":["Able to sample applications including one or more parallel paradigms from the following:","* MPI (node-level)","* OpenMP","* Pthreads","Able to sample applications and nodes including at least one specialized form of measurement from the following:","* I\/O performance","* Hardware counters","* Perf metrics","* Linux tracepoints","Able to use Lo2s in application monitoring and system monitoring mode"],"maintainer":["* Mario Bielert, ZIH Tools Team @ TU Dresden"],"exams":{"questions":0}},"pe\/2\/3\/3\/b":{"title":"PE2.3.3 Vampir","id":"pe\/2\/3\/3\/b","learning outcomes":["Able to launch Vampir both stand-alone and connected to a VampirServer instance if available","Able to use the function summary to determine at a high level what parts of the code may not perform well","Able to use the summary timeline to quickly identify changed behavior over time and determine next steps for investigation","Able to identify when a small number of processes have divergent behavior from the rest of the application","Able to use the performance radar and counter timelines to correlate performance metrics with the code that is executing","Able to use the communication matrix to describe communication patterns and identify potential bottlenecks and\/or imbalances"],"maintainer":["* Bert Wesarg, William (Bill) Williams, ZIH Tools Team @ TU Dresden"],"exams":{"questions":0}},"pe\/2\/3\/7\/b":{"title":"PE2.3.7 NVIDIA Nsight Systems","id":"pe\/2\/3\/7\/b","learning outcomes":["* Use the CLI to identify common optimization targets","* Generate traces for analysis in the GUI","* Understand how to use the GUI to find inefficient parts in an application"],"maintainer":["* Markus Velten, ZIH Team @ TU Dresden"],"exams":{"questions":0}},"pe\/2\/3\/4\/b":{"title":"PE2.3.4 Darshan","id":"pe\/2\/3\/4\/b","learning outcomes":["* Understand Darshan"],"exams":{"questions":0}},"pe\/b":{"error":"skill is incomplete","id":"pe\/b"},"pe\/5\/b":{"title":"PE5 Optimization Cycle","id":"pe\/5\/b","learning outcomes":["Apply the full workflow for tuning a parallel program."],"exams":{"questions":0}},"pe\/1\/2\/b":{"title":"PE1.2 Total Cost of Ownership","id":"pe\/1\/2\/b","requirements":[],"learning outcomes":["1. **Define the concept** of Total Cost of Ownership and its relevance to performance engineering.","2. **Identify all components** that contribute to the TCO of engineering projects, including hidden and indirect costs.","3. **Calculate TCO** for various engineering assets using structured financial models.","4. **Analyze case studies** to illustrate the impact of TCO on project decision-making and long-term sustainability.","5. **Evaluate different purchasing strategies** and their effects on TCO, including lease vs. buy decisions.","6. **Assess the influence** of maintenance and operational practices on the TCO of assets.","7. **Compare TCO** across different technology solutions to determine the most cost-effective choice.","8. **Develop a strategic approach** to minimize TCO through efficient design, procurement, and asset management techniques.","9. **Utilize software tools** and frameworks to automate TCO calculations and provide actionable insights.","10. **Forecast long-term costs** and benefits associated with different project options to guide strategic planning.","11. **Interpret financial reports** and data to enhance TCO-based decision-making.","12. **Synthesize TCO analysis** with environmental and social governance (ESG) considerations to promote sustainable practices.","13. **Discuss the challenges** of accurate TCO estimation in complex and uncertain environments.","14. **Propose innovative solutions** to reduce TCO while maintaining or enhancing performance and compliance.","15. **Critique existing TCO models** and methodologies to identify areas for improvement.","16. **Explore future trends** in cost management and how they might influence TCO calculations in performance engineering.","17. **Implement risk assessment strategies** to mitigate financial uncertainties in TCO evaluations.","18. **Examine regulatory impacts** on TCO, including compliance costs and potential financial incentives.","19. **Engage with experts** through seminars and workshops to gain industry insights on TCO management.","20. **Design project scenarios** to apply TCO calculations in real-world engineering challenges.","21. **Lead discussions** on ethical considerations in cost reporting and TCO manipulation.","AI generated content"],"exams":{"questions":0}},"pe\/1\/b":{"title":"PE1 Cost Awareness","id":"pe\/1\/b","requirements":[],"learning outcomes":["* Identify many of the constraint for the time to get the solution.","* Infer the total cost of ownership."],"subskills":["pe\/1\/1\/b","pe\/1\/2\/b"],"exams":{"questions":0}},"pe\/1\/1\/b":{"title":"PE1.1 Time to Solution Constraints","id":"pe\/1\/1\/b","requirements":[],"learning outcomes":["1. **Remember the fundamental concepts** and definitions related to time to solution constraints in performance engineering.","2. **Understand the implications** of various constraints on project timelines and outcomes.","3. **Apply standard methods** and tools to measure and evaluate time to solution in different scenarios.","4. **Analyze data** to identify critical constraints that adversely affect project timelines.","5. **Evaluate the effectiveness** of different strategies for managing time to solution constraints in project environments.","6. **Create innovative solutions** and optimization strategies to effectively reduce time to solution while maintaining quality and budget considerations.","7. **Summarize key factors** and metrics that influence time to solution in engineering projects.","8. **Interpret project data and reports** to forecast potential delays and prepare mitigation strategies.","9. **Design a framework** for continuous improvement in managing time to solution constraints across multiple projects.","10. **Assess the impact** of implemented solutions on overall project efficiency and cost reduction.","11. **Formulate plans** to integrate time to solution constraints into broader project scopes and long-term strategies.","12. **Synthesize various approaches** for overcoming specific time to solution challenges in complex projects.","13. **Critique existing methodologies** and propose modifications to enhance efficiency and effectiveness in achieving desired outcomes.","14. **Construct models** to simulate different scenarios and predict the effects of various time to solution strategies.","15. **Discuss the ethical considerations** in time management, emphasizing transparency and accountability in project reporting and stakeholder communications.","16. **Explore emerging tools and technologies** that could revolutionize time to solution strategies in performance engineering.","AI generated content"],"exams":{"questions":0}},"pe\/3\/2\/b":{"title":"PE3.2 Controlled Experiments","id":"pe\/3\/2\/b","requirements":[],"learning outcomes":["* Differentiate types of benchmarks:","* The Linpack benchmark is used, for example, to build the TOP 500 list of the currently fastest supercomputers, which is updated twice a year.","* For HPC users, however, synthetic tests to benchmark HPC cluster hardware (like the Linpack benchmark) are of less importance, because the emphasis lies on the determination of speedups and efficiencies of the parallel program they want to use.","* Comprehend that benchmarking is very essential in the HPC environment and can be applied to a variety of issues:","* What is the scalability of my program?","* How many cluster nodes can be maximally used, before the efficiency drops to values which are unacceptable?","* How does the same program perform in different cluster environments?","* Comprehend that benchmarking is also a basis for dealing with questions emerging from tuning, e.g.:","* What is the appropriate task size (big vs. small) that may have a positive performance impact on my program?","* Is the use of hyper-threading technology advantageous?","* What is the best mapping of processes to nodes, pinning of processes\/threads to CPUs or cores, and setting memory affinities to NUMA nodes in order to speed up a parallel program?","* What is the best compiler selection for my program (GCC, Intel, PGI, ...), in combination with the most suitable MPI environment (Open MPI, Intel MPI, ...)?","* What is the best compiler generation\/version for my program?","* What are the best compiler options regarding, for example, the optimization level -O2, -O3, . . . , for building the executable program?","* Is the use of PGO (Profile Guided Optimization) or other high-level optimization, e.g. using IPA\/IPO (Inter-Procedural Analyzer\/Inter-Procedural Optimizer), helpful?","* What is the performance behavior after a (parallel) algorithm has been improved, i.e. to what extent are speedup, efficiency, and scalability improved?","* Assess speedups and efficiencies as the key measures for benchmarks of a parallel program.","* Benchmark the runtime behaviour of parallel programs, performing controlled experiments by providing varying HPC resources (e.g. 1, 2, 4, 8, ... cores on shared memory systems or 1, 2, 4, 8, ... nodes on distributed systems for the benchmarks).","* Measure runtimes by the help of tools like:","* Built-in time command, e.g. for MPI programs ('time mpirun ... my-mpi-app').","* Stand-alone 'time' program, e.g. for sequential or OpenMP programs ('\/usr\/bin\/time my-openmp-app')."],"exams":{"questions":0}},"pe\/3\/b":{"title":"PE3 Benchmarking","id":"pe\/3\/b","learning outcomes":["* Produce a design and the required documentation for a benchmark.","* Compose a controlled experiment from a design and measure important values.","* Apply both strong and weak scaling to a benchmark to find out which is the appropriate model."],"subskills":["pe\/3\/1\/b","pe\/3\/2\/b","pe\/3\/3\/b"],"exams":{"questions":0}},"pe\/3\/1\/b":{"title":"PE3.1 Design & Documentation","id":"pe\/3\/1\/b","requirements":[],"learning outcomes":["* **Understand the role** of design and documentation in the context of performance engineering.","* **Develop comprehensive design documents** that clearly outline system architecture and performance criteria.","* **Create detailed documentation** that supports the reproducibility and verification of performance benchmarks.","* **Apply best practices** in technical writing to enhance clarity and precision in documentation.","* **Utilize diagrams and flowcharts** to visually represent system designs and benchmarking processes.","* **Implement version control systems** to manage changes in design documents and maintain historical records.","* **Coordinate with multidisciplinary teams** to gather input and ensure all aspects of system performance are documented.","* **Evaluate existing documentation** to identify gaps in information and areas for improvement.","* **Integrate feedback mechanisms** into documentation practices to continually refine and improve design clarity.","* **Train team members** on effective documentation techniques and standards.","* **Review and update documentation** regularly to reflect system upgrades and changes.","* **Develop templates and guidelines** for performance engineering documentation to standardize practices across projects.","* **Discuss the ethical implications** of documentation in maintaining transparency and accountability in engineering projects.","* **Explore software tools** that aid in the documentation and design process, such as CAD for system layouts and benchmarking software for performance analysis.","* **Simulate documentation audits** to prepare for compliance checks and internal reviews.","* **Lead workshops** on effective design and documentation strategies within the organization.","* **Analyze case studies** where effective documentation has led to successful benchmarking and system optimizations.","* **Implement a feedback loop** from field performance data to refine initial designs and documentation.","* **Develop a comprehensive understanding** of the regulatory requirements impacting design documentation in different industries.","* **Craft a documentation strategy** that aligns with organizational goals and performance benchmarks.","* **Master technical communication skills** to convey complex performance data in understandable formats.","* **Assess the impact of design decisions** on system performance through detailed documentation reviews.","* **Incorporate sustainability considerations** into system design documentation.","* **Facilitate effective knowledge transfer** through well-organized documentation practices.","* **Develop performance narratives** that align technical documentation with stakeholder needs and expectations.","AI generated content"],"exams":{"questions":0}},"pe\/3\/3\/b":{"title":"PE3.3 Strong vs. Weak Scaling","id":"pe\/3\/3\/b","requirements":[],"learning outcomes":["* Differentiate types of scaling:","* **Weak scaling**: problem size increases proportionally to the number of parallel processes to analyze how big may the problems be that I can solve.","* **Strong scaling**: problem size remains the same for an increasing number of processes to analyze how fast can I solve a problem of a given size.","* Interpret typical weak and strong scaling plots.","* Avoid typical pitfalls:","* **Break-even considerations regarding the benchmark effort**:","* Benchmarking also represents a certain effort, namely for providing the HPC resources and human time explicitly used for that purpose.","* **Presenting fair speedups**:","* For conventional speedup calculations the same version of an algorithm (the same program) is used to measure runtimes T(sequential) and T(parallel) but for fair speedup calculations, the best known sequential algorithm to measure T(sequential) should be used.","* **Special features of current CPU architectures**:","* Features like turbo boost and hyper-threading may influence benchmark results.","* **Shared nodes**:","* If the same cores are potentially shared at times on a node by different programs, the value of the benchmark results may be significantly reduced or even made uselessly.","* **Reproducibility**:","* There are parallel algorithms which may produce non-deterministic results, due to inherent effects of concurrency which in turn may lead to different (but generally equivalent) results but also to strongly differing runtimes of repeated runs."],"exams":{"questions":0}},"pe\/4\/2\/b":{"title":"PE4.2 Tuning without Modifying the Source Code","id":"pe\/4\/2\/b","requirements":[],"learning outcomes":["* Understand Small vs. Big Tasks","* Understand Process Mapping to Nodes","* Understand CPU and Thread Pinning","* Understand Memory Affinity (NUMA)","* Understand Optimized Libraries","* Understand Compiler Options \/ Optimization Switches","* Understand Profile Guided Optimization Workflow (PGO)","* Understand Package Specific Options","* Understand Runtime Options for MPI and OpenMP"],"exams":{"questions":0}},"pe\/4\/b":{"title":"PE4 Tuning","id":"pe\/4\/b","learning outcomes":["* Analyze the IO performance of a program and tune it to increase performance.","* Use profiling tools such as prof, gprof and line-by-line profilers.","* Define Amdahl's Law.","* Tune a parallel program in order to achieve better runtimes and to optimize the usage of the HPC resources."],"subskills":["pe\/4\/1\/b","pe\/4\/2\/b","pe\/4\/3\/b"],"exams":{"questions":0}},"pe\/4\/1\/b":{"title":"PE4.1 I\/O Tuning","id":"pe\/4\/1\/b","requirements":[],"learning outcomes":["* **Understand the fundamentals** of I\/O processes and the impact on system performance.","* **Identify common I\/O bottlenecks** in both hardware and software and strategies to mitigate them.","* **Implement I\/O monitoring tools** to track performance issues and assess the effectiveness of tuning efforts.","* **Optimize file system configurations** to enhance data throughput and reduce latency.","* **Adjust operating system settings** to better align with the specific I\/O demands of applications.","* **Design and execute benchmarks** to measure I\/O performance before and after tuning interventions.","* **Apply advanced tuning techniques** such as RAID configurations, SSD caching, and file system optimization.","* **Evaluate different storage technologies** (e.g., HDD vs. SSD) and their configurations for optimal I\/O performance.","* **Develop a systematic approach** to continuous I\/O performance assessment and tuning.","* **Synthesize I\/O tuning practices with overall system performance management** to ensure holistic optimization.","* **Explore the use of software-defined storage solutions** for flexible I\/O performance enhancement.","* **Discuss the impact of network configurations** on I\/O performance, particularly in distributed systems.","* **Incorporate security considerations** into I\/O tuning practices to protect data integrity.","* **Utilize virtualization technologies** to simulate different I\/O scenarios and their impacts on system performance.","* **Lead teams in collaborative projects** that focus on optimizing I\/O operations across different departments.","* **Stay informed about the latest developments** in I\/O technology and tuning methodologies.","* **Critique existing I\/O tuning methods** and propose innovative solutions based on up-to-date research.","* **Develop best practices documentation** for I\/O tuning that can be shared within the organization.","* **Host workshops and training sessions** to disseminate I\/O tuning knowledge and practices.","* **Evaluate the environmental impact** of I\/O tuning decisions, considering energy consumption and sustainability.","* **Master the use of diagnostic tools** for identifying and resolving I\/O issues.","* **Examine the effects of file system choice** on I\/O performance, including comparisons of file systems under different workloads.","* **Facilitate effective data management strategies** to complement I\/O tuning efforts.","* **Integrate I\/O performance metrics** into overall system health indicators.","* **Train technical staff** on troubleshooting I\/O problems quickly and efficiently.","AI generated content"],"exams":{"questions":0}},"pe\/4\/3\/b":{"title":"PE4.3 Tuning via Reprogramming","id":"pe\/4\/3\/b","requirements":[],"learning outcomes":["* Understand Efficient Algorithms","* Understand Functional Units","* Understand Vectorization, SIMD","* Understand SIMT"],"exams":{"questions":0}}}