Czytanie tekstu z (Facebook) RecyclerView with AccessibilityService

Jak mogę zobaczyć tekst nieostre dzieci z RecyclerView (przedmioty)?Czytanie tekstu z (Facebook) RecyclerView with AccessibilityService

Moja usługa dostępności ma odczytać tekst z ekranu zawierającego słowa kluczowe. Działa to dobrze w większości sytuacji, z wyjątkiem sytuacji, gdy mam do czynienia z RecyclerViews, szczególnie z aplikacją Facebook.

Oba ekrany komentarzy na Facebooku i komentarzy są oparte na programie RecyclerView i żadne z nich nie zapewnia dostępu do potrzebnego tekstu ze swoich zdarzeń ułatwień dostępu.

Co próbowałem:

public void onAccessibilityEvent(AccessibilityEvent event) { 

    //I've tried looping through event.getText() and it does not contain the text entries 

    //I've also tried the following (with various null checks) 
    AccessibilityRecordCompat record = AccessibilityEventCompat.asRecord(event); 
    AccessibilityNodeInfoCompat source = record.getSource(); 
    List<AccessibilityNodeInfoCompat> nodes = source.findAccessibilityNodeInfosByText("..."); 
    //(and loop through the entries) 

    //I've also tried limiting to the class name "android.support.v7.widget.RecyclerView" 
    //and looking at parcelableData 

    //In the facebook app, the following is always null too 
    source.findFocus(AccessibilityNodeInfoCompat.FOCUS_ACCESSIBILITY); 

}

Nigdzie nie wydaje się zawierać dane mi potrzeba. Jako przykład podczas otwierania ekranu Komentarz/nakładkę z postu, mogę zobaczyć całą zawartość tekstową z wyjątkiem samych komentarzach (które są wewnątrz widoku recyklera)

Dzienniki szczegółów zdarzeń, które widzę:

Oto, co widzę na nakładce uwagi na stanowisku z jednym komentarzu

Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.support.v7.widget.RecyclerView; Text: []; ContentDescription: null; ItemCount: 100; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: true; BeforeText: null; FromIndex: 13; ToIndex: 24; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_STATE_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: X.BGT; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: true; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.support.v7.widget.RecyclerView; Text: []; ContentDescription: null; ItemCount: 1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: 0; ToIndex: 0; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.view.ViewGroup; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: Photo; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.ImageButton; Text: []; ContentDescription: Stickers; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.TextView; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0 
Event: EventType: TYPE_WINDOW_STATE_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: X.BGT; Text: [Be the first to like this., Like Button, Photo Comment, Write a comment…, Open GIF search, Stickers]; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: true; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0

ja również próbował odkrywania dzieci źródła, ale tylko ViewGroups są przedstawiane (brak Obejrzane) i zawsze z tekstem null

Ancestor for 0 = [email protected]; boundsInParent: Rect(0, 0 - 1056, 236); boundsInScreen: Rect(12, 244 - 1068, 480); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0 = [email protected]; boundsInParent: Rect(0, 0 - 1056, 236); boundsInScreen: Rect(12, 244 - 1068, 480); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 0 = [email protected]; boundsInParent: Rect(0, 0 - 828, 58); boundsInScreen: Rect(204, 262 - 1032, 320); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 0, 0 = [email protected]; boundsInParent: Rect(0, 0 - 268, 58); boundsInScreen: Rect(204, 262 - 472, 320); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 1 = [email protected]; boundsInParent: Rect(0, 0 - 828, 48); boundsInScreen: Rect(204, 320 - 1032, 368); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 2 = [email protected]; boundsInParent: Rect(0, 0 - 828, 80); boundsInScreen: Rect(204, 382 - 1032, 462); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 2, 0 = [email protected]; boundsInParent: Rect(0, 0 - 93, 44); boundsInScreen: Rect(507, 394 - 600, 438); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null] 
Ancestor for 0, 0, 2, 1 = [email protected]; boundsInParent: Rect(0, 0 - 114, 44); boundsInScreen: Rect(632, 394 - 746, 438); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]

Nie jestem pewien, czy jest to zdecydowanie możliwe, czy też całkowicie niewłaściwe podejście. Zauważyłem, że w trybie TalkBack użytkownik musi kliknąć/skupić poszczególne elementy dziecka z RecyclerView, zanim zostaną one przeczytane, co nie jest opcją dla mojej usługi (która ma znaleźć te interesujące przedmioty)

Źródło

2017-10-25 Nick Cardoso

Mam 23 godziny do przyznania nagrody za to pytanie. Jeśli ktoś ma choćby mgliste pojęcie, które mogłoby mu pomóc, chciałbym to zobaczyć. –

Czy osiągnąłeś jakieś miejsce w odniesieniu do rozwiązania? – ashishdhiman2007

Nie, przepraszam –

Mam do czynienia problem z zagnieżdżonymi widokami przewijania, rozwiązany przez rekursywne znajdowanie wszystkich widoków, myślę, że już to wypróbowałeś. Oto rozwiązanie działa w moim przypadku, możesz wypróbować to może pracować również dla Ciebie.

private AccessibilityNodeInfo accessbiltyContent(AccessibilityNodeInfo parentView,String strAddress) throws Exception { 

    int childCount = parentView.getChildCount(); 


    for (int i = 0; i < childCount; ++i) { 
     AccessibilityNodeInfo child = parentView.getChild(i); 
     // child=null; 
     if (child != null) { 

      String strToAddress=strAddress+" > "+child.getClassName().toString(); 
      String nodeText=""; 

      if(child.getText() != null) 
       nodeText=child.getText().toString(); 


      Log.v(TAG,"values =" +child.getClassName().toString(),nodeText,strToAddress)); 
      accessbiltyContent(child,strToAddress); 

     } 
     else { 
      return null; 
     } 
    } 

    return null; 


}

Przejdzie przez wszystkie widoki w rodzicu i wydrukuje je w celu znalezienia.

Źródło

2017-10-31 15:24:18

Tak, to nic nie da - wszystkie węzły tekstowe będą miały wartość zerową.Zasadniczo jest to ten sam kod, który wygenerował mój ostatni zestaw logów w pytaniu: –

@Mamidi Anil Kumar: Czy próbowałeś pobierać posty z FB przy pomocy Accessibility API, czy byłeś w stanie pobrać tekst postów? – ashishdhiman2007

Czytanie tekstu z (Facebook) RecyclerView with AccessibilityService

Odpowiedz

Powiązane problemy