Rysowanie przebiegu za pomocą AVAssetReader

81

Czytam utwór z biblioteki iPoda za pomocą assetUrl (w kodzie o nazwie audioUrl) Mogę go odtworzyć na wiele sposobów, mogę go wyciąć, mogę to zrobić, ale ... Naprawdę nie rozumiem, co z tym zrobię CMSampleBufferRef, aby uzyskać dane do rysowania przebiegu! Potrzebuję informacji o wartościach szczytowych, jak mogę to uzyskać w ten (może inny) sposób?

    AVAssetTrack * songTrack = [audioUrl.tracks objectAtIndex:0];
    AVAssetReaderTrackOutput * output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:nil];
    [reader addOutput:output];
    [output release];

    NSMutableData * fullSongData = [[NSMutableData alloc] init];
    [reader startReading];

    while (reader.status == AVAssetReaderStatusReading){

        AVAssetReaderTrackOutput * trackOutput = 
        (AVAssetReaderTrackOutput *)[reader.outputs objectAtIndex:0];

        CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];

        if (sampleBufferRef){/* what I gonna do with this? */}

Proszę pomóż mi!

iFreeman
źródło

Odpowiedzi:

261

Szukałem podobnej rzeczy i postanowiłem „skręcić własną”. Zdaję sobie sprawę, że to stary post, ale na wypadek, gdyby ktoś inny go szukał, oto moje rozwiązanie. jest stosunkowo szybki i brudny oraz normalizuje obraz do „pełnej skali”. obrazy, które tworzy, są „szerokie”, tzn. musisz umieścić je w UIScrollView lub w inny sposób zarządzać wyświetlaniem.

jest to oparte na niektórych odpowiedziach udzielonych na to pytanie

Przykładowe wyjście

przykładowy przebieg

EDYCJA: Dodałem logarytmiczną wersję metod uśredniania i renderowania, zobacz koniec tego komunikatu dla alternatywnych wersji i wyników porównania. Osobiście wolę oryginalną wersję liniową, ale zdecydowałem się ją opublikować, na wypadek gdyby ktoś mógł ulepszyć zastosowany algorytm.

Będziesz potrzebować tych importów:

#import <MediaPlayer/MediaPlayer.h>
#import <AVFoundation/AVFoundation.h>

Najpierw ogólna metoda renderowania, która pobiera wskaźnik do uśrednionych danych przykładowych
i zwraca UIImage. Uwaga: te próbki nie są odtwarzalnymi próbkami audio.

-(UIImage *) audioImageGraph:(SInt16 *) samples
                normalizeMax:(SInt16) normalizeMax
                 sampleCount:(NSInteger) sampleCount 
                channelCount:(NSInteger) channelCount
                 imageHeight:(float) imageHeight {

    CGSize imageSize = CGSizeMake(sampleCount, imageHeight);
    UIGraphicsBeginImageContext(imageSize);
    CGContextRef context = UIGraphicsGetCurrentContext();

    CGContextSetFillColorWithColor(context, [UIColor blackColor].CGColor);
    CGContextSetAlpha(context,1.0);
    CGRect rect;
    rect.size = imageSize;
    rect.origin.x = 0;
    rect.origin.y = 0;

    CGColorRef leftcolor = [[UIColor whiteColor] CGColor];
    CGColorRef rightcolor = [[UIColor redColor] CGColor];

    CGContextFillRect(context, rect);

    CGContextSetLineWidth(context, 1.0);

    float halfGraphHeight = (imageHeight / 2) / (float) channelCount ;
    float centerLeft = halfGraphHeight;
    float centerRight = (halfGraphHeight*3) ; 
    float sampleAdjustmentFactor = (imageHeight/ (float) channelCount) / (float) normalizeMax;

    for (NSInteger intSample = 0 ; intSample < sampleCount ; intSample ++ ) {
        SInt16 left = *samples++;
        float pixels = (float) left;
        pixels *= sampleAdjustmentFactor;
        CGContextMoveToPoint(context, intSample, centerLeft-pixels);
        CGContextAddLineToPoint(context, intSample, centerLeft+pixels);
        CGContextSetStrokeColorWithColor(context, leftcolor);
        CGContextStrokePath(context);

        if (channelCount==2) {
            SInt16 right = *samples++;
            float pixels = (float) right;
            pixels *= sampleAdjustmentFactor;
            CGContextMoveToPoint(context, intSample, centerRight - pixels);
            CGContextAddLineToPoint(context, intSample, centerRight + pixels);
            CGContextSetStrokeColorWithColor(context, rightcolor);
            CGContextStrokePath(context); 
        }
    }

    // Create new image
    UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();

    // Tidy up
    UIGraphicsEndImageContext();   

    return newImage;
}

Następnie metoda, która pobiera AVURLAsset i zwraca dane obrazu PNG

- (NSData *) renderPNGAudioPictogramForAsset:(AVURLAsset *)songAsset {

    NSError * error = nil;
    AVAssetReader * reader = [[AVAssetReader alloc] initWithAsset:songAsset error:&error];
    AVAssetTrack * songTrack = [songAsset.tracks objectAtIndex:0];

    NSDictionary* outputSettingsDict = [[NSDictionary alloc] initWithObjectsAndKeys:
                                        [NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
                                        //     [NSNumber numberWithInt:44100.0],AVSampleRateKey, /*Not Supported*/
                                        //     [NSNumber numberWithInt: 2],AVNumberOfChannelsKey,    /*Not Supported*/
                                        [NSNumber numberWithInt:16],AVLinearPCMBitDepthKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,
                                        nil];

    AVAssetReaderTrackOutput* output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict];

    [reader addOutput:output];
    [output release];

    UInt32 sampleRate,channelCount;

    NSArray* formatDesc = songTrack.formatDescriptions;
    for(unsigned int i = 0; i < [formatDesc count]; ++i) {
        CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[formatDesc objectAtIndex:i];
        const AudioStreamBasicDescription* fmtDesc = CMAudioFormatDescriptionGetStreamBasicDescription (item);
        if(fmtDesc ) {

            sampleRate = fmtDesc->mSampleRate;
            channelCount = fmtDesc->mChannelsPerFrame;

            //    NSLog(@"channels:%u, bytes/packet: %u, sampleRate %f",fmtDesc->mChannelsPerFrame, fmtDesc->mBytesPerPacket,fmtDesc->mSampleRate);
        }
    }

    UInt32 bytesPerSample = 2 * channelCount;
    SInt16 normalizeMax = 0;

    NSMutableData * fullSongData = [[NSMutableData alloc] init];
    [reader startReading];

    UInt64 totalBytes = 0;         
    SInt64 totalLeft = 0;
    SInt64 totalRight = 0;
    NSInteger sampleTally = 0;

    NSInteger samplesPerPixel = sampleRate / 50;

    while (reader.status == AVAssetReaderStatusReading){

        AVAssetReaderTrackOutput * trackOutput = (AVAssetReaderTrackOutput *)[reader.outputs objectAtIndex:0];
        CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];

        if (sampleBufferRef){
            CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef);

            size_t length = CMBlockBufferGetDataLength(blockBufferRef);
            totalBytes += length;

            NSAutoreleasePool *wader = [[NSAutoreleasePool alloc] init];

            NSMutableData * data = [NSMutableData dataWithLength:length];
            CMBlockBufferCopyDataBytes(blockBufferRef, 0, length, data.mutableBytes);

            SInt16 * samples = (SInt16 *) data.mutableBytes;
            int sampleCount = length / bytesPerSample;
            for (int i = 0; i < sampleCount ; i ++) {

                SInt16 left = *samples++;
                totalLeft  += left;

                SInt16 right;
                if (channelCount==2) {
                    right = *samples++;
                    totalRight += right;
                }

                sampleTally++;

                if (sampleTally > samplesPerPixel) {

                    left  = totalLeft / sampleTally; 

                    SInt16 fix = abs(left);
                    if (fix > normalizeMax) {
                        normalizeMax = fix;
                    }

                    [fullSongData appendBytes:&left length:sizeof(left)];

                    if (channelCount==2) {
                        right = totalRight / sampleTally; 

                        SInt16 fix = abs(right);
                        if (fix > normalizeMax) {
                            normalizeMax = fix;
                        }

                        [fullSongData appendBytes:&right length:sizeof(right)];
                    }

                    totalLeft   = 0;
                    totalRight  = 0;
                    sampleTally = 0;
                }
            }

           [wader drain];

            CMSampleBufferInvalidate(sampleBufferRef);
            CFRelease(sampleBufferRef);
        }
    }

    NSData * finalData = nil;

    if (reader.status == AVAssetReaderStatusFailed || reader.status == AVAssetReaderStatusUnknown){
        // Something went wrong. return nil

        return nil;
    }

    if (reader.status == AVAssetReaderStatusCompleted){

        NSLog(@"rendering output graphics using normalizeMax %d",normalizeMax);

        UIImage *test = [self audioImageGraph:(SInt16 *) 
                         fullSongData.bytes 
                                 normalizeMax:normalizeMax 
                                  sampleCount:fullSongData.length / 4 
                                 channelCount:2
                                  imageHeight:100];

        finalData = imageToData(test);
    }        

    [fullSongData release];
    [reader release];

    return finalData;
}

Opcja zaawansowana: Wreszcie, jeśli chcesz móc odtwarzać dźwięk za pomocą AVAudioPlayer, musisz buforować go w folderze pamięci podręcznej pakietów aplikacji. Ponieważ to robiłem, zdecydowałem się również buforować dane obrazu i opakowałem całość w kategorię UIImage. musisz dołączyć tę ofertę open source, aby wyodrębnić dźwięk i trochę kodu z tego miejsca, aby obsłużyć niektóre funkcje wątkowania w tle.

najpierw niektóre definicje i kilka ogólnych metod klas do obsługi nazw ścieżek itp

//#define imgExt @"jpg"
//#define imageToData(x) UIImageJPEGRepresentation(x,4)

#define imgExt @"png"
#define imageToData(x) UIImagePNGRepresentation(x)

+ (NSString *) assetCacheFolder  {
    NSArray  *assetFolderRoot = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES);
    return [NSString stringWithFormat:@"%@/audio", [assetFolderRoot objectAtIndex:0]];
 }

+ (NSString *) cachedAudioPictogramPathForMPMediaItem:(MPMediaItem*) item {
    NSString *assetFolder = [[self class] assetCacheFolder];
    NSNumber * libraryId = [item valueForProperty:MPMediaItemPropertyPersistentID];
    NSString *assetPictogramFilename = [NSString stringWithFormat:@"asset_%@.%@",libraryId,imgExt];
    return [NSString stringWithFormat:@"%@/%@", assetFolder, assetPictogramFilename];
}

+ (NSString *) cachedAudioFilepathForMPMediaItem:(MPMediaItem*) item {
    NSString *assetFolder = [[self class] assetCacheFolder];

    NSURL    * assetURL = [item valueForProperty:MPMediaItemPropertyAssetURL];
    NSNumber * libraryId = [item valueForProperty:MPMediaItemPropertyPersistentID];

    NSString *assetFileExt = [[[assetURL path] lastPathComponent] pathExtension];
    NSString *assetFilename = [NSString stringWithFormat:@"asset_%@.%@",libraryId,assetFileExt];
    return [NSString stringWithFormat:@"%@/%@", assetFolder, assetFilename];
}

+ (NSURL *) cachedAudioURLForMPMediaItem:(MPMediaItem*) item {
    NSString *assetFilepath = [[self class] cachedAudioFilepathForMPMediaItem:item];
    return [NSURL fileURLWithPath:assetFilepath];
}

Teraz metoda inicjująca, która wykonuje „biznes”

- (id) initWithMPMediaItem:(MPMediaItem*) item 
           completionBlock:(void (^)(UIImage* delayedImagePreparation))completionBlock  {

    NSFileManager *fman = [NSFileManager defaultManager];
    NSString *assetPictogramFilepath = [[self class] cachedAudioPictogramPathForMPMediaItem:item];

    if ([fman fileExistsAtPath:assetPictogramFilepath]) {

        NSLog(@"Returning cached waveform pictogram: %@",[assetPictogramFilepath lastPathComponent]);

        self = [self initWithContentsOfFile:assetPictogramFilepath];
        return self;
    }

    NSString *assetFilepath = [[self class] cachedAudioFilepathForMPMediaItem:item];

    NSURL *assetFileURL = [NSURL fileURLWithPath:assetFilepath];

    if ([fman fileExistsAtPath:assetFilepath]) {

        NSLog(@"scanning cached audio data to create UIImage file: %@",[assetFilepath lastPathComponent]);

        [assetFileURL retain];
        [assetPictogramFilepath retain];

        [NSThread MCSM_performBlockInBackground: ^{

            AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:assetFileURL options:nil];
            NSData *waveFormData = [self renderPNGAudioPictogramForAsset:asset]; 

            [waveFormData writeToFile:assetPictogramFilepath atomically:YES];

            [assetFileURL release];
            [assetPictogramFilepath release];

            if (completionBlock) {

                [waveFormData retain];
                [NSThread MCSM_performBlockOnMainThread:^{

                    UIImage *result = [UIImage imageWithData:waveFormData];

                    NSLog(@"returning rendered pictogram on main thread (%d bytes %@ data in UIImage %0.0f x %0.0f pixels)",waveFormData.length,[imgExt uppercaseString],result.size.width,result.size.height);

                    completionBlock(result);

                    [waveFormData release];
                }];
            }
        }];

        return nil;

    } else {

        NSString *assetFolder = [[self class] assetCacheFolder];

        [fman createDirectoryAtPath:assetFolder withIntermediateDirectories:YES attributes:nil error:nil];

        NSLog(@"Preparing to import audio asset data %@",[assetFilepath lastPathComponent]);

        [assetPictogramFilepath retain];
        [assetFileURL retain];

        TSLibraryImport* import = [[TSLibraryImport alloc] init];
        NSURL    * assetURL = [item valueForProperty:MPMediaItemPropertyAssetURL];

         [import importAsset:assetURL toURL:assetFileURL completionBlock:^(TSLibraryImport* import) {
            //check the status and error properties of
            //TSLibraryImport

            if (import.error) {

                NSLog (@"audio data import failed:%@",import.error);

            } else{
                NSLog (@"Creating waveform pictogram file: %@", [assetPictogramFilepath lastPathComponent]);
                AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:assetFileURL options:nil];
                NSData *waveFormData = [self renderPNGAudioPictogramForAsset:asset]; 

                [waveFormData writeToFile:assetPictogramFilepath atomically:YES];

                if (completionBlock) {
                     [waveFormData retain];
                    [NSThread MCSM_performBlockOnMainThread:^{

                        UIImage *result = [UIImage imageWithData:waveFormData];
                        NSLog(@"returning rendered pictogram on main thread (%d bytes %@ data in UIImage %0.0f x %0.0f pixels)",waveFormData.length,[imgExt uppercaseString],result.size.width,result.size.height);

                        completionBlock(result);

                        [waveFormData release];
                    }];
                }
            }

            [assetPictogramFilepath release];
            [assetFileURL release];

        }  ];

        return nil;
    }
}

Przykład wywołania tego:

-(void) importMediaItem {

    MPMediaItem* item = [self mediaItem];

    // since we will be needing this for playback, save the url to the cached audio.
    [url release];
    url = [[UIImage cachedAudioURLForMPMediaItem:item] retain];

    [waveFormImage release];

    waveFormImage = [[UIImage alloc ] initWithMPMediaItem:item completionBlock:^(UIImage* delayedImagePreparation){

        waveFormImage = [delayedImagePreparation retain];
        [self displayWaveFormImage];
    }];

    if (waveFormImage) {
        [waveFormImage retain];
        [self displayWaveFormImage];
    }
}

Logarytmiczna wersja metod uśredniania i renderowania

#define absX(x) (x<0?0-x:x)
#define minMaxX(x,mn,mx) (x<=mn?mn:(x>=mx?mx:x))
#define noiseFloor (-90.0)
#define decibel(amplitude) (20.0 * log10(absX(amplitude)/32767.0))

-(UIImage *) audioImageLogGraph:(Float32 *) samples
                normalizeMax:(Float32) normalizeMax
                 sampleCount:(NSInteger) sampleCount 
                channelCount:(NSInteger) channelCount
                 imageHeight:(float) imageHeight {

    CGSize imageSize = CGSizeMake(sampleCount, imageHeight);
    UIGraphicsBeginImageContext(imageSize);
    CGContextRef context = UIGraphicsGetCurrentContext();

    CGContextSetFillColorWithColor(context, [UIColor blackColor].CGColor);
    CGContextSetAlpha(context,1.0);
    CGRect rect;
    rect.size = imageSize;
    rect.origin.x = 0;
    rect.origin.y = 0;

    CGColorRef leftcolor = [[UIColor whiteColor] CGColor];
    CGColorRef rightcolor = [[UIColor redColor] CGColor];

    CGContextFillRect(context, rect);

    CGContextSetLineWidth(context, 1.0);

    float halfGraphHeight = (imageHeight / 2) / (float) channelCount ;
    float centerLeft = halfGraphHeight;
    float centerRight = (halfGraphHeight*3) ; 
    float sampleAdjustmentFactor = (imageHeight/ (float) channelCount) / (normalizeMax - noiseFloor) / 2;

    for (NSInteger intSample = 0 ; intSample < sampleCount ; intSample ++ ) {
        Float32 left = *samples++;
        float pixels = (left - noiseFloor) * sampleAdjustmentFactor;
        CGContextMoveToPoint(context, intSample, centerLeft-pixels);
        CGContextAddLineToPoint(context, intSample, centerLeft+pixels);
        CGContextSetStrokeColorWithColor(context, leftcolor);
        CGContextStrokePath(context);

        if (channelCount==2) {
            Float32 right = *samples++;
            float pixels = (right - noiseFloor) * sampleAdjustmentFactor;
            CGContextMoveToPoint(context, intSample, centerRight - pixels);
            CGContextAddLineToPoint(context, intSample, centerRight + pixels);
            CGContextSetStrokeColorWithColor(context, rightcolor);
            CGContextStrokePath(context); 
        }
    }

    // Create new image
    UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();

    // Tidy up
    UIGraphicsEndImageContext();   

    return newImage;
}

- (NSData *) renderPNGAudioPictogramLogForAsset:(AVURLAsset *)songAsset {

    NSError * error = nil;
    AVAssetReader * reader = [[AVAssetReader alloc] initWithAsset:songAsset error:&error];
    AVAssetTrack * songTrack = [songAsset.tracks objectAtIndex:0];

    NSDictionary* outputSettingsDict = [[NSDictionary alloc] initWithObjectsAndKeys:
                                        [NSNumber numberWithInt:kAudioFormatLinearPCM],AVFormatIDKey,
                                        //     [NSNumber numberWithInt:44100.0],AVSampleRateKey, /*Not Supported*/
                                        //     [NSNumber numberWithInt: 2],AVNumberOfChannelsKey,    /*Not Supported*/

                                        [NSNumber numberWithInt:16],AVLinearPCMBitDepthKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsBigEndianKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsFloatKey,
                                        [NSNumber numberWithBool:NO],AVLinearPCMIsNonInterleaved,
                                        nil];

    AVAssetReaderTrackOutput* output = [[AVAssetReaderTrackOutput alloc] initWithTrack:songTrack outputSettings:outputSettingsDict];

    [reader addOutput:output];
    [output release];

    UInt32 sampleRate,channelCount;

    NSArray* formatDesc = songTrack.formatDescriptions;
    for(unsigned int i = 0; i < [formatDesc count]; ++i) {
        CMAudioFormatDescriptionRef item = (CMAudioFormatDescriptionRef)[formatDesc objectAtIndex:i];
        const AudioStreamBasicDescription* fmtDesc = CMAudioFormatDescriptionGetStreamBasicDescription (item);
        if(fmtDesc ) {

            sampleRate = fmtDesc->mSampleRate;
            channelCount = fmtDesc->mChannelsPerFrame;

            //    NSLog(@"channels:%u, bytes/packet: %u, sampleRate %f",fmtDesc->mChannelsPerFrame, fmtDesc->mBytesPerPacket,fmtDesc->mSampleRate);
        }
    }

    UInt32 bytesPerSample = 2 * channelCount;
    Float32 normalizeMax = noiseFloor;
    NSLog(@"normalizeMax = %f",normalizeMax);
    NSMutableData * fullSongData = [[NSMutableData alloc] init];
    [reader startReading];

    UInt64 totalBytes = 0; 
    Float64 totalLeft = 0;
    Float64 totalRight = 0;
    Float32 sampleTally = 0;

    NSInteger samplesPerPixel = sampleRate / 50;

    while (reader.status == AVAssetReaderStatusReading){

        AVAssetReaderTrackOutput * trackOutput = (AVAssetReaderTrackOutput *)[reader.outputs objectAtIndex:0];
        CMSampleBufferRef sampleBufferRef = [trackOutput copyNextSampleBuffer];

        if (sampleBufferRef){
            CMBlockBufferRef blockBufferRef = CMSampleBufferGetDataBuffer(sampleBufferRef);

            size_t length = CMBlockBufferGetDataLength(blockBufferRef);
            totalBytes += length;

            NSAutoreleasePool *wader = [[NSAutoreleasePool alloc] init];

            NSMutableData * data = [NSMutableData dataWithLength:length];
            CMBlockBufferCopyDataBytes(blockBufferRef, 0, length, data.mutableBytes);

            SInt16 * samples = (SInt16 *) data.mutableBytes;
            int sampleCount = length / bytesPerSample;
            for (int i = 0; i < sampleCount ; i ++) {

                Float32 left = (Float32) *samples++;
                left = decibel(left);
                left = minMaxX(left,noiseFloor,0);
                totalLeft  += left;

                Float32 right;
                if (channelCount==2) {
                    right = (Float32) *samples++;
                    right = decibel(right);
                    right = minMaxX(right,noiseFloor,0);
                    totalRight += right;
                }

                sampleTally++;

                if (sampleTally > samplesPerPixel) {

                    left  = totalLeft / sampleTally; 
                    if (left > normalizeMax) {
                        normalizeMax = left;
                    }

                   // NSLog(@"left average = %f, normalizeMax = %f",left,normalizeMax);

                    [fullSongData appendBytes:&left length:sizeof(left)];

                    if (channelCount==2) {
                        right = totalRight / sampleTally; 

                        if (right > normalizeMax) {
                            normalizeMax = right;
                        }

                        [fullSongData appendBytes:&right length:sizeof(right)];
                    }

                    totalLeft   = 0;
                    totalRight  = 0;
                    sampleTally = 0;
                }
            }

           [wader drain];

            CMSampleBufferInvalidate(sampleBufferRef);
            CFRelease(sampleBufferRef);
        }
    }

    NSData * finalData = nil;

    if (reader.status == AVAssetReaderStatusFailed || reader.status == AVAssetReaderStatusUnknown){
        // Something went wrong. Handle it.
    }

    if (reader.status == AVAssetReaderStatusCompleted){
        // You're done. It worked.

        NSLog(@"rendering output graphics using normalizeMax %f",normalizeMax);

        UIImage *test = [self audioImageLogGraph:(Float32 *) fullSongData.bytes 
                                 normalizeMax:normalizeMax 
                                  sampleCount:fullSongData.length / (sizeof(Float32) * 2) 
                                 channelCount:2
                                  imageHeight:100];

        finalData = imageToData(test);
    }

    [fullSongData release];
    [reader release];

    return finalData;
}

wyniki porównania

Liniowy
Fabuła liniowa na początek filmu „Warm It Up” firmy Acme Swing

logarytmiczny
Wykres logarytmiczny na początku filmu „Warm It Up” firmy Acme Swing

niezsynchronizowane
źródło
22
To bardzo kompletna i użyteczna odpowiedź. W rzeczywistości jest to samouczek z pogranicza, który możesz rozważyć umieszczenie na blogu lub czymś podobnym. Głosowałbym na ciebie 10, gdybym mógł.
Daniel G. Wilson
3
Tak, naprawdę powinieneś napisać samouczek lub wpis na blogu ... z przykładowym projektem;)
Daniel
2
Nie mam pojęcia, jakie to szybkie, ale działa! Musiałem dodać / zmodyfikować kilka rzeczy, ponieważ zakodowałeś na stałe kilka założeń, takich jak 2-kanałowy dźwięk. Znalazłem również następujące przydatne do obliczania samplePerPixel (sposób, w jaki przypisałeś, który wydawał się arbitralny? / 50 ??) NSTimeInterval duration = (float) songAsset.duration.value / (float) songAsset.duration.timescale; NSLog (@ "Czas trwania nagrania:% f sek.", Czas trwania);
podkowa7
Chcę również zaktualizować go pod kątem grafiki siatkówki i prawdopodobnie przekształcić metodę UIImage * w metodę drawRect: i mieć możliwość dodania podświetlonego regionu. DZIĘKUJEMY za wspaniały punkt wyjścia. AVFoundation i biblioteki niższego poziomu wciąż są trochę przerażające dla stosunkowo niedoświadczonych
horseshoe7
3
Dzięki za to. Użyłem tego jako punktu wyjścia dla kontrolki kakao, która dodaje kilka innych funkcji, takich jak wyświetlanie postępu gry - źródło w źródle github.com/fulldecent/FDWaveformView
William Entriken
5

Powinieneś być w stanie pobrać bufor audio z twojego sampleBuffRef, a następnie iterować po tych wartościach, aby zbudować swój przebieg:

CMBlockBufferRef buffer = CMSampleBufferGetDataBuffer( sampleBufferRef );
CMItemCount numSamplesInBuffer = CMSampleBufferGetNumSamples(sampleBufferRef);
AudioBufferList audioBufferList;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
                                                            sampleBufferRef,
                                                            NULL,
                                                            &audioBufferList,
                                                            sizeof(audioBufferList),
                                                            NULL,
                                                            NULL,
                                                                  kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
                                                            &buffer
                                                            );

// this copies your audio out to a temp buffer but you should be able to iterate through this buffer instead
SInt32* readBuffer = (SInt32 *)malloc(numSamplesInBuffer * sizeof(SInt32));
memcpy( readBuffer, audioBufferList.mBuffers[0].mData, numSamplesInBuffer*sizeof(SInt32));
Joel
źródło
3

Inne podejście wykorzystujące Swift 5 i AVAudioFile:

///Gets the audio file from an URL, downsaples and draws into the sound layer.
func drawSoundWave(fromURL url:URL, fromPosition:Int64, totalSeconds:UInt32, samplesSecond:CGFloat) throws{

    print("\(logClassName) Drawing sound from \(url)")

    do{
        waveViewInfo.samplesSeconds = samplesSecond

        //Get audio file and format from URL
        let audioFile = try AVAudioFile(forReading: url)

        waveViewInfo.format = audioFile.processingFormat
        audioFile.framePosition = fromPosition * Int64(waveViewInfo.format.sampleRate)

        //Getting the buffer
        let frameCapacity:UInt32 = totalSeconds * UInt32(waveViewInfo.format.sampleRate)

        guard let audioPCMBuffer = AVAudioPCMBuffer(pcmFormat: waveViewInfo.format, frameCapacity: frameCapacity) else{ throw AppError("Unable to get the AVAudioPCMBuffer") }
        try audioFile.read(into: audioPCMBuffer, frameCount: frameCapacity)
        let audioPCMBufferFloatValues:[Float] = Array(UnsafeBufferPointer(start: audioPCMBuffer.floatChannelData?.pointee,
                                                                          count: Int(audioPCMBuffer.frameLength)))

        waveViewInfo.points = []
        waveViewInfo.maxValue = 0
        for index in stride(from: 0, to: audioPCMBufferFloatValues.count, by: Int(audioFile.fileFormat.sampleRate) / Int(waveViewInfo.samplesSeconds)){

            let aSample = CGFloat(audioPCMBufferFloatValues[index])
            waveViewInfo.points.append(aSample)
            let fix = abs(aSample)
            if fix > waveViewInfo.maxValue{
                waveViewInfo.maxValue = fix
            }

        }

        print("\(logClassName) Finished the points - Count = \(waveViewInfo.points.count) / Max = \(waveViewInfo.maxValue)")

        populateSoundImageView(with: waveViewInfo)

    }
    catch{

        throw error

    }

}

///Converts the sound wave in to a UIImage
func populateSoundImageView(with waveViewInfo:WaveViewInfo){

    let imageSize:CGSize = CGSize(width: CGFloat(waveViewInfo.points.count),//CGFloat(waveViewInfo.points.count) * waveViewInfo.sampleSpace,
                                  height: frame.height)
    let drawingRect = CGRect(origin: .zero, size: imageSize)

    UIGraphicsBeginImageContextWithOptions(imageSize, false, 0)
    defer {
        UIGraphicsEndImageContext()
    }
    print("\(logClassName) Converting sound view in rect \(drawingRect)")

    guard let context:CGContext = UIGraphicsGetCurrentContext() else{ return }

    context.setFillColor(waveViewInfo.backgroundColor.cgColor)
    context.setAlpha(1.0)
    context.fill(drawingRect)
    context.setLineWidth(1.0)
    //        context.setLineWidth(waveViewInfo.lineWidth)

    let sampleAdjustFactor = imageSize.height / waveViewInfo.maxValue
    for pointIndex in waveViewInfo.points.indices{

        let pixel = waveViewInfo.points[pointIndex] * sampleAdjustFactor

        context.move(to: CGPoint(x: CGFloat(pointIndex), y: middleY - pixel))
        context.addLine(to: CGPoint(x: CGFloat(pointIndex), y: middleY + pixel))

        context.setStrokeColor(waveViewInfo.strokeColor.cgColor)
        context.strokePath()

    }

     //        for pointIndex in waveViewInfo.points.indices{
    //
    //            let pixel = waveViewInfo.points[pointIndex] * sampleAdjustFactor
    //
    //            context.move(to: CGPoint(x: CGFloat(pointIndex) * waveViewInfo.sampleSpace, y: middleY - pixel))
    //            context.addLine(to: CGPoint(x: CGFloat(pointIndex) * waveViewInfo.sampleSpace, y: middleY + pixel))
    //
    //            context.setStrokeColor(waveViewInfo.strokeColor.cgColor)
    //            context.strokePath()
    //
    //        }

    //        var xIncrement:CGFloat = 0
    //        for point in waveViewInfo.points{
    //
    //            let normalizedPoint = point * sampleAdjustFactor
    //
    //            context.move(to: CGPoint(x: xIncrement, y: middleY - normalizedPoint))
    //            context.addLine(to: CGPoint(x: xIncrement, y: middleX + normalizedPoint))
    //            context.setStrokeColor(waveViewInfo.strokeColor.cgColor)
    //            context.strokePath()
    //
    //            xIncrement += waveViewInfo.sampleSpace
    //
    //        }

    guard let soundWaveImage = UIGraphicsGetImageFromCurrentImageContext() else{ return }

    soundWaveImageView.image = soundWaveImage
    //        //In case of handling sample space in for
    //        updateWidthConstraintValue(soundWaveImage.size.width)
    updateWidthConstraintValue(soundWaveImage.size.width * waveViewInfo.sampleSpace)

}

GDZIE

class WaveViewInfo {

    var format:AVAudioFormat!
    var samplesSeconds:CGFloat = 50
    var lineWidth:CGFloat = 0.20
    var sampleSpace:CGFloat = 0.20

    var strokeColor:UIColor = .red
    var backgroundColor:UIColor = .clear

    var maxValue:CGFloat = 0
    var points:[CGFloat] = [CGFloat]()

}

W tej chwili drukuje tylko jedną falę dźwiękową, ale można ją przedłużyć. Dobre jest to, że możesz wydrukować ścieżkę dźwiękową według części

Reimond Hill
źródło
a co z transmisją audio na żywo?
mojtaba al moussawi
podejście, ale bądź inny. Najlepszym podejściem byłoby wypełnienie bufora danych i narysowanie go, ale to mój gość.
Reimond Hill