Introduction to DataStage Director - Data Warehousing Data Warehousing

In the Datastage Director, we can:

run, schedule and monitor jobs
view job status, logs and schedules
filter the displayed events

Click on the datastage director icon to open the application:

director icon

Fill out the server details, user credentials and choose the project name

The DataStage Director window is divided into two panes:

The Job Category pane lists all of the jobs in the repository.
The right pane shows one of three views: Status view, Schedule view, or Log view.

This table describes DataStage Director menu options:

Menu Option	Description
Project	Open another project, print, or exit.
View	Display or hide the toolbar, status bar, buttons, or job category pane, specify sorting order, change views, filter entries, show more details, or refresh the screen.
Search	Start a text search dialog box.
Job	Validate, run, schedule, stop, or reset a job, purge old entries from the job log file, delete unwanted jobs, clean up job resources (if this is enabled), set default job parameter values.
Tools	Monitor running jobs, manage job batches, start the DataStage Designer.
Help	Displays online help.

DataStage Director has three view options:

The Status view displays the status, date and time started, elapsed time, and other run information about each job in the selected repository category.
The Schedule view displays job scheduling details.
The Log view displays all of the events for a particular run of a job.

To check for job completions, these can be checked in Status column ( Complied, Aborted, Finished, etc.). The start time and end times are also listed in the director

TO see summary of a particular job run, double click the job and the below window with job parameters, status, etc will pop up

job summary datastage director

For debugging, we would need to look at the detailed log. click on the log icon

The log would have info records, warnings and Fatal errors that would help in debugging the issues.

Meet Ananth Tirumanur. Hi there 👋

I work on projects in data science, big data, data engineering, data modeling, software engineering, and system design.

👨‍💻 All of my projects are available at https://github.com/akrish1982
💬 Ask me about Data engineering, SQL, Databases, Data pipelines, Data infrastructure
📄 My work history: https://www.linkedin.com/in/ananthtirumanur/
⚡ Fun fact: Marathoner & Casual Birding enthusiast

Connect with me:

Twitter: @akrish82
LinkedIn: https://linkedin.com/in/ananthtirumanur/

My Resources:

LinkedIn Newsletter: Data Engineering with AWS
Udemy Course: AWS Certified Data Engineer Associate Practice Test
Python Crash Course: Python Crash Course on Gumroad

Languages and Tools:

AWS, Bash, Docker, Elasticsearch, Git, Grafana, Hadoop, Hive, EMR, Glue, Athena, Lambda, Step Functions, Airflow/MWAA, DynamoDB, Kafka, Kubernetes, Linux, MariaDB, MySQL, Pandas, PostgreSQL, Python, Redis, Scala, SQLite