scout24 berlin Building a Central Hive Metastore in a Multi-Account AWS Cloud Environment | Raffael Dzikowski

Cloud
Buy a ticket for the Data Festival 2019 at: https://www.datafestival.de/tickets/

One of the central challenges when providing a Data Lake architecture in a multi-account Cloud setup on AWS consists in offering a consistent, familiar, and easy-to-use access mechanism to it which ideally hides all the nitty-gritty details of varying data formats and data sources, while at the same time making the data available for analysis in a convenient way. A proven abstraction that meets these requirements is a set of relations (tables) that enable analyses using the well-established Structured Query Language (SQL). In order to to accomplish this, a metastore is necessary that will manage all the table data in one central place. This talk will be about how we build such a central metastore which can be accessed by an arbitrary number of Elastic MapReduce clusters, spread over many accounts belonging to the company and running different kinds of analysis workloads. We will go into technical details of the implementation and also show the benefits of such a solution from the perspective of different stake holders.

Raffael Dzikowski, Big Data Engineer, Scout24 Group

scout24 berlin Building a Central Hive Metastore in a Multi-Account AWS Cloud Environment | Raffael Dzikowski
3 Likes	3 Dislikes
65 views views	49 followers
Science & Technology	Upload TimePublished on 27 Feb 2019

Related keywords

cloud nine,leadership quotes,scout24 share price,scout24 wiki,video phone beyonce mp3,scout24 logo,video phone commax,videos for cats,leadership principles amazon,leadership styles,scout24 offer,video phone letra,free ringtones,freedom mortgage,airpods 2,video doorbell,munich pact,video phone number,aia vitality,sharing amazon prime,sharingan eyes,video phone é a continuação de telephone,video player,sharing community,munich reinsurance,camera iphone x apk,airspace,airport,munich oktoberfest,business letter format,big data conference,cloudy with a chance of meatballs,sharing is caring,münchen pronounce,munich conference,munich germany weather,business license,big data analysis,data science salario,business plan template,camera iphone 7 vs iphone 8,video editing app,business analyst,big data university,free music,scout24 market cap,upload video to youtube,video phone ft lady gaga,data science from scratch,freep,big data definition,video phone extended remix,video phone call,sharingbox,leadership high school,munich weather,video editor,münchen deutschland,münchen wetter,cloud bread,aia mpf,cloud gate,business for sale,camera iphone,camera phone comparison,sharing is caring meme,upload resume to linkedin,sharing wifi password,münchen flughafen terminal,app video iphone,sharing wifi iphone,camera iphone 8 plus,data science porto alegre,camera iphone xr,münchen city,munich re,scout24 ipo,upload music to spotify,scout24 stock,business name generator,scout24 stock price,free people,sharing outlook calendar,sharing network 5k,scout24 ag,big data engineer salary,upload soundcloud,data science para negócios,münchen germany,leadership definition,cloud 9,video phone beyonce letra,business administration,münchen in english,cloudy urine,free movies,airport authority,big data dangerous,camera phone x apk,münchen sehenswürdigkeiten,münchen zob,sharingan contacts,video games,business cards,video phone lyrics,upload to instagram from pc,münchen flughafen,munich time,airsim,video star,data science curso,cloudy with a chance of meatballs 2,free games,airbnb hk,leadership synonym,camera phone x,video phone tradução,leadership books,cloud computing,scout24 blackstone,video converter,cloud types,video phone lady gaga,video maker,business analyst salary,cloud print,video phone feat lady gaga,video editing software,data science pucrs,free credit report,free credit score,camera iphone 8 plus apk,big data in healthcare,business insider,camera iphone x vs xr,munich airport,video to gif,business casual women,big data examples,air canada,big data technologies,camera phone ranking,münchen airport,clouded leopard,free robux,big data companies,upload to imgur,big data interview questions,data science unisinos,leadership public schools,videoder,videographer,data science para negócios pdf,camera phone x - os 12 camera apk,big data hadoop,münchener rückversicherungs-gesellschaft aktiengesellschaft,sharing shelf,leader definition,data science do zero,upload speed,camera iphone 7 plus,scout24 consumer services,video camera,munich movie,cloudflare,leaderboard fedex,camera iphone apk,leaderboard,free dating sites,cloud couch,aia,aimer,free stock photos,scout24 news,aig,data science for business,uploadvr,upload photos to icloud,data science para leigos,data science academy vale a pena,data science course,munich olympics,business casual men,upload speed test,upload photo,video downloader,scout24 investor relations,leader synonym,münchener tierpark hellabrunn,air supply,münchen meaning,munich to prague,video phone beyonce,sharingan naruto,upload download speed,cloudberry,munich airport map,münchen hbf,scout24 annual report,münchener rückversicherungs-gesellschaft,business card holder,business loans,upload youtube,upload to google drive,freeform,munich re careers,business insurance,camera iphone xr vs xs,videogamedunkey,munich to berlin,münchener hypothekenbank eg,sharing economy,münchener rückversicherungs-gesellschaft ag,sharing kindle books,scout24 elliott,scout24 cars,sharing community yonkers,free vpn,camera iphone x,video editing software free,free fonts,video phone screen,upload pictures,munich to vienna,data science academy python,leader of the pack,airbnb,leadership academy,cloudland canyon,free online games,munich agreement,cloudy pool water,free movies online,video phone wikipedia,data science academy é bom,freecell,video phone gaga,video phone beyonce lyrics,business plan,data science brasil,cloud eggs,freebirds,scout24 capital markets day,video killed the radio star,business professional,upload pdf to kindle,air seoul,video game stores,munich hotels,business casual,upload studio,freestyle libre,munich re america,sharing network,cloud strife,münchen hauptbahnhof,cloudera,camera iphone 8,upload image,big data band,upload file,airasia,data science uniritter,big data tools,camera iphone 7,camera iphone 6,big data architecture,leadership,data science do zero pdf,munich germany,big data engineer,leadership principles,big data certification,camera phone 2019,leadership interview questions,aims,sharing google calendar,big data analytics,airdrop,data science academy power bi,aims cityu,data science brazil,business ideas,scout24 agm,upload video,scout24 press release,sharingan,leaders eat last,sharingan tattoo,video phone beyonce lady gaga,leadership skills,business credit cards,video to mp3,cloud atlas,münchen university,airpods,big data dangerous lyrics,leadership traits,big data jobs,