Working with Digital Data

in Religious Studies

13. Advanced Processing & AI: Using AI in Your Data Pipeline

Summer Semester 2024
Prof. Dr. Nathan Gibson

Outline

1. Presentations
2. Review: Audio & Video
3. Tutorial: Video Processing –Break–
4. AI Pipeline
5. Evaluation

Presentations

5 minutes:

1. What is the format and source of your data? (Include critical reflection.)
2. How did you edit, process, filter, or add to it? Why? (Show how you have used at least one of the approaches we discussed in class.)
3. What is the most interesting question you might answer with your dataset?
4. What is the most valuable thing you learned in the process?

📈 Review: Processing Audio & Video

🧭 Objective: Prepare audio and video for analysis with an appropriate workflow and tools.

📈 Review: Processing Audio & Video: Considerations

• sources (digitized/not digitized)
• format/length
• quality

📈 Review: Processing Audio & Video: Analysis Goals

• What aspects of the audio/video does your analysis relate to? Words/text, music/sound, visual, connection between these?

📈 Review: Processing Audio & Video: Target Data & Metadata Formats

What information do you need to generate or tag to do this analysis? What format do the media files ultimately need to be in?

• logs of files?
• timestamps of scenes?
• frame-grabs?
• encoding?

📈 Review: Processing Audio & Video: Tools

Two especially important tools:

• WhisperAI for transcribing audio
• FFMPEG: command-line utility for converting and sampling video and audio

Tutorial: Video Processing

https://24data.pages.gwdg.de/video-processing-tutorial

Break

AI Pipeline

AI Pipeline: Example

AI-generated image “alt” captions

AI Pipeline

Your turn (see whiteboard)

Evaluation

http://r.sd.uni-frankfurt.de/372cb30c

Preview

Advanced Processing & AI: Use AI to Label Your Data