Join The Expert Series group and get advice from the industry experts!
SharePoint Cleanup Best Practices
In this episode the Experts will delve into the best practices for SharePoint Cleanup, provide their expertise to your questions, demonstrate valuable tools such as SharePoint Essentials Toolkit and provide Free Expert Scripts (PowerShell) to help you!
Webinar Transcript
QiPoint’s The Expert Series Intro | August 2, 2107
Breaking News!! Many companies today have major investments in the SharePoint and Office 365 intranets, and now they want to make the most of it. QiPoint has assembled some of the best and brightest MVPs in the industry to help answer your questions.
Presenting to you…The Expert Series! We will discuss the hottest topics and provide tips, tricks and best practices for SharePoint and Office 365.
Animated Host: Thanks, Dave. Some topics that these experts will cover include best practices using electronic workflows, advice on cleaning up your SharePoint environment, help with the latest Office 365 apps, help with migration tasks (such as fixing broken links). Don’t miss out on these free and valuable webinars. Sign up now.
EPISODE 01: SharePoint Cleanup Best Practices Discussion
Recorded on August 2, 2017 (2:00 PM EST). Time: 60 minutes.
Presenter (Chris Ang): … I’m the SharePoint architect at QiPoint, and I’m really excited that everyone’s here to join us. I’m super excited that we’ve got Andrew, Liam, Joel, and Vlad on the line to give their knowledge and decades of experience to give us some advice on this topic. My contact info is below on the screen and should also be in the invite, but let me know if you have any questions regarding our product. Near the end, there will be a link to get to The Expert Series home page (qipoint.com/blog/experts). It is also in the email invitation that you should have received. You’ll be able to get updates there as well—some PowerShell scripts and an infographic at the end in a couple of days from today I will be posting as well as this video recording. We will be having a webinar tomorrow as well as next week on Tuesday for a follow-up for this series, and it will have a demonstration of one of the products like QiPoint to do clean-up.
I’m going to go over first the agenda of what we’ll be covering. I’ll make some introductions of the experts for those who may need to learn a little bit about who we have here. We’ll cover than the expert advice. Basically, it’s going to be in a question-and-answer format, so we’ll have different topics around development for Andrew, infrastructure for Vlad. We’ll have that section of the webinar we’ll be covering the questions and answers. We won’t be able to answer every single question out there regarding SharePoint clean-up, but we’ve gathered some of the most frequently asked questions nowadays that these experts have been getting and sharing them with you. During the webinar, there is a question section in your GoToWebinar panel. You should be able to see that. You can post questions that you would like to ask the expert. For all of the episodes of The Expert Series, you’ll be able to ask these questions and have direct access to these experts to get answers.
Let me just jump in right now for a second and just say that anytime during the webinar, you can put in your question in the panel, and those questions will be what we are going to build our collateral and content and will basically be the questions we answer. I’d highly recommend this: If you hear something that we are talking about and it spurs up ideas, feel free to put those questions in. We definitely don’t want empty time, so we definitely welcome your questions. After we go through some of the questions and answers with the experts that we have the canned questions and answers, there’ll be a short demo and will have a time for answering the questions that you’ll be posting.
Let me go to the introductions. As you heard just now, the voice of Joel Oleson. Joel, many of you may know of already. I sure have heard of him long before I actually met him. I’d say that I’ve been working with SharePoint a long time, but knowing these days can say that as Joel can he was the first SharePoint administrator ever. In the year 2000, Microsoft had released a product called “Tahoe.” I think it was called in and later became known as SharePoint. Joel was part of the product team there and was the first SharePoint administrator ever, so he’s made lots of contribution and shared a lot of knowledge. Way back then there was very little knowledge that you could find on SharePoint and help, and it’s really gotten a lot better over the years, thanks to people like Joel who have helped us out a lot.
Going through these introductions, and I don’t want to take too much time on it, but I wanted to ask each of the experts to give us one thing that no one knows about them that they could share that might be interesting. I know Joel, you’re a world traveler. You went to over 150 countries in your lifetime so far. Is there anything else you’d like to share that is interesting that no one knows about you?
Joel Oleson: Yea! Yea! Some people may not of realized that in my travels I’ve had some pretty crazy experiences. One of the ones that happened when I was in Vietnam. It was a SharePoint Saturday. It was a good time. After the event, we had a speaker dinner, and they kind of singled me out. They ended up cutting up this cobra and cutting out his heart. One of the cultural things there was to give the guest of honor a beating cobra heart. So, I ended up shooting with some with a drink this beating cobra heart. It was pretty crazy. In fact, Dux (Dux Raymond Si) got a video of it. I’ll have to dig that up at some point.
Chris Ang: Wow! That’s pretty intense.
Joel Oleson: Yeah!
Chris Ang: Thanks, Joel. So, we also have Liam, SharePoint security guru, so any questions relating to SharePoint security you can direct at Liam. He’s an 11-year SharePoint MVP, and I’ve read a lot of articles from Liam. Liam, thanks for being here. You want to share something with the group here.
Liam Cleary: Yeah, obviously I’m a true geek at heart. One of the things that I’m quite of doing is that I have a bunch of Raspberry Pis in my basement and I built a Raspberry Pi cluster in my house and purely based on wanting to say that I’d done it more than anything else. It serves no other purpose in my house right now, but I actually have one.
Chris Ang: Everyone needs one of those in their house.
Liam Cleary: Correct!
Joel Oleson: One of the things I love about Liam is his hacking SharePoint sessions. He was one of the first that kind of said, ‘Let me show you how to hack SharePoint.’ I love that he kind of puts it on his head to show you how to secure SharePoint by showing you how it easy it is to hack it when people do the wrong thing.
Liam Cleary: …but we won’t be doing that today.
Chris Ang: We’ll do that another day. Okay, thanks Liam. We have Andrew, our SharePoint development guru. Again, I’ve followed Andrew for many years. My background is in development and he’s always had excellent articles and contributions, as well. You’ve probably seen him as well at Ignite and conferences. He’s also a 12-year SharePoint MVP. Thank you, Andrew, so much for being here. I really appreciate it. I heard that you started a SharePoint training company recently. That sounds exciting. I think a lot of people could benefit from that. Is there anything that you’d like to share with the team and anything you’ve been working on and maybe your training business that you started.
Andrew Connell: Sure, thanks for having me, Chris. I started a tiny business about a year ago. Don’t have any courses out yet, but they will be on the SharePoint framework for developers and targeting early release later this month. I guess an interesting fact about me and nothing technology-wise, but going back many generations, there’s a guy named Daniel who was a great liberator of Ireland who is a great-great-great-great grandfather of mine, so I’ve got a statue of a relative at the center of O’Connell Street in Dublin, Ireland.
Chris Ang: Awesome! Wow! One of the greats like yourself, too.
Andrew Connell: …moving people away from the developments ….
Chris Ang: What’s interesting about Andrew is that he’s been doing training forever. I think he was one of the first SharePoint trainers, working way, way, way back. Some of the first SharePoint training stuff. I remember meeting Andrew at some early training classes.
Andrew Connell: I’ll poke fun at myself. When you can’t do, you teach. So there you go.
Chris Ang: Thanks Andrew. It’s great to have you, too. Last but not least we’ve got Vlad, our IT Pro expert and infrastructure. I know I’ve used there’s an article I know Vlad’s ruined many but maximizing SQL Server for SharePoint. I used that for many years, and it was great. Thanks to Vlad for everything and your contributions and for being here today. Is there anything you want to tell us about yourself, dark secrets or something?
Vlad Catrinescu: Hi Chris. Really happy to be there. Let’s see, the thing I have prepared. You all told me that I had to find something really funny about myself and then you’re all being super cool, epic travel adventure stuff. It’s not fair. The thing I was going to say is something funny. The only one I think one of the panelists today that is not from actual the US and in Quebec there’s a lot of like fast food and a lot of things that we don’t have that you guys have in the US so one of the things that I always have to do when I travel to the US is hit up either a Nando’s actually or a 7-11 to get Big Gulp. There you go. On the more serious side and to be like cool like the other guys here. I think I was one of the youngest SharePoint MVPs out there. I got my MVP when I was 21, so I don’t know if that was a good thing or a bad thing. I got the SharePoint designation when I was young.
Chris Ang: Wow! Might be a record. Great. Alright. Thanks Vlad. Really great to have you here, too. OK, let’s try to get to questions, but first this poll is going to help the experts understand what type of audience we have so it will help them gear their answers. OK, so the first one is, What SharePoint platform(s) to you use? We got SharePoint 2016, but probably not a lot of people on that yet, but 2013, 2010 and older. If you can share, that would be great so that you can help them understand what type of audience we have out there for these clean-up questions that we have.
Joel Oleson: Yeah, and by the way, you can definitely select multiple. I know there are some hybrid environments out there, and the idea is to just pick and choose the ones that if you have older versions and newer versions, pick and choose.
Chris Ang: Right.
Joel Oleson: It looks like we are about there. 2013 is definitely the largest section here.
Chris Ang: Yep.
Joel Oleson: Go for it.
Chris Ang: So, you should be able to see this poll results, so we’ve got a good portion on SharePoint 2013. Almost more than I saw on 2016 and a good portion on SharePoint Online and some on 2010. Great. I want to hurry up and get to these answers. What everyone is here for. The first one is in the area of development for Andrew.
(Time: 12:33)
Development Cleanup (Andrew Connell)
Question 1: We are cleaning up our custom code and third party WSP components installed. What would you suggest we do when deciding the best approach?
Christ Ang: The first question we have here for you, Andrew, is that a lot of companies will have custom code and third-party WSPs. Sometimes it’s required and sometimes they can cause problems. When a company needs to clean up or thinking about cleaning up the custom code and WSP components installed, what is your recommendation on best approach to cleaning up WSPs and custom code.
Andrew Connell: Let me rephrase the questions to make it more what I want to say, but just to give it a little more context around it so when I think “clean-up” and what people are asking from a development point of view is that we’ve been doing things in our legacy way, like you said, doing WSPs and server-side code. What’s the right way to move forward whether you’re going to be on-prem or whether you’re going to be on Office 365. I think that the best approach to try to default to with all of the stuff that you do that’s extensibility-wise as far as SharePoint goes is to build like you’re going to the cloud, because SharePoint is predominantly a cloud product that we then get essentially a snapshots that will run on-prem. If you build like you’re going to Office 365, then everything you build could potentially make it to Office 365, but it will also work on-prem. Later this year, we are going to see a feature pack come out for SharePoint 2016. I’d expect that we will see that in September at Ignite. That’s when they will probably announcement it. That is just me reading the tea leaves and Microsoft taking advantage of a big event at Ignite in Orlando be able to do that. I think the best approach is that once they do that if you are on SharePoint 2016, then you’ll be able to focus on the things that you build and on the new framework that we’re given to use for extensibility. That’s the SharePoint framework, where you build client-side solutions, and you get a first-class deployment model to ship stuff up to SharePoint, either on-prem or in the cloud. Now, that doesn’t help people who are still on SharePoint 2013, and I know we had a significant portion of those people that answered that poll and said they were on SharePoint 2013, and of course, some are still on 2010 or even older that—2007. For you, I apologize. I mean, I empathize. The approach I would take in terms of cleaning up is that I think that anything you build you want client << >> to be a client-side solution type thing, something that ideally is going to run in the browser, so JavaScript, HTL, CSS images to build your solution. Now I know there are a lot of traditional SharePoint developers out there that are going to roll their eyes and say, ‘Not interested in going there. I like my server-side code. That’s fine, but the idea is that you’re trying to build in the spirit of things not running or hosted by SharePoint and instead they are running in the client. If you have to and don’t want to go client-side and use server-side code, I would try to build it to where it runs either beside or outside of SharePoint. So you’re not going to use the SharePoint server side object model. You’re not going to be building certain WSPs and features and stuff like that. Instead what you are going to be focusing on is building something that’s going to run outside or external to SharePoint that either a client-side solution is going to be able to call from SharePoint, so something hosted on AWS or an Azure or Google cloud platform. If you’re doing event handler-type stuff, we have that capability using your functions and certainly all of the different service technologies that we have. I guess to sum it up is that I think that whatever you have if you’re trying to clean up stuff and trying to move forward, you’re going to have to rebuild the things that you’ve built as WSPs and running server side. You’re going to want that stuff either first try to make it run client side. Not everything is going to be able to run client side and that everything is going to get damage to the SharePoint framework, so then you’re next fallback is to build it so it can run outside of SharePoint and talk to SharePoint using the different APIs and SDKs that they have that Microsoft just made available to us using the client object model or using the Rest APIs.
Chris Ang: Great! Yeah, I think I went from server-side code and going to see some wasn’t actually as bad as I thought. Thanks. So, what you are saying and recommending is to move to a client-side model as much as possible.
Andrew Connell: Yeah. I would say that’s going to give you your easiest transition to taking advantage of future versions of SharePoint. Microsoft clearly does not want to host your server-side code inside SharePoint or an Office 365 anymore. The more you try to do that the more you are just bucking the trend of trying to push a rock up a hill. It’s not going to happen. That’s not Microsoft’s business model. That’s not Microsoft’s interest. I think it’s better to adjust what you’re doing if you still want to extend SharePoint to develop on top of SharePoint and so to that respect, embrace what they are trying to do—pushing this off the client; pushing this off the server; pushing us to external hosting environments like Azure, like AWS, like GCP or like your own, if you’re on-prem, like an Xbox, like Azure stack. There are plenty of options for doing it but if you’re building a DLL. If you’re deploying to SharePoint, you’re essentially deploying to a brick wall. There’s only so far you’re doing to be able to go, and Microsoft is making no investments in that area.
Joel Oleson: I want to jump in here for a second. This is Joel. I’ve noticed that a lot of people are trying to get out of their custom-code solutions they built on-prem. If they look at it in an evaluation process and when those things were built it was a buy vs. build and they decided to build. If you evaluate buy vs. build in relation to cloud, there’s a lot. There’s thousands of solutions and add-ins. I don’t want to name any by name, but as an example, one of the very popular ones is Nintex. Nintex has got your forms and workflows and, as an example, there are hundreds and thousands of those types of solutions if you are trying to orchestrate something or trying to build a form, you’re going to Nintex and K2s. There are a ton of people out there who have built a lot of the heavy lifting for you. Some people just need to start over and say, ‘hey, maybe there’s a framework (that actually may come up later—the word “framework”), but I think that idea—buy vs. build—you may want to just start over with that idea of buy vs. build.
Chris Ang: Right. That’s a good point. I’m biased because we sell those third-party products, but yes, I think it’s a good option to look at. I want to keep moving because I want to make sure that we get to all of the questions here, but thanks, Andrew. That kind of leads into the second question for you.
(Time: 20:00)
Development Cleanup (Andrew Connell)
Question #2: Should we convert old server side code to use the new SharePoint framework? What are the top advantages?
Chris Ang: If coming to the question of “I’ve got my old server-side code why would I want to go to the SharePoint framework. What are the top (maybe not all of the advantages) but maybe the top few or three or your top advantages why you would suggest moving to the SharePoint framework that would benefit me over finance? It’s more economical or support-wise or what are the advantages?
Andrew Connell: I think the biggest on is this is where Microsoft is moving and where they are investing everything. The big thing there, all of the extensibility piece that they are doing, SharePoint online and Office 365, and One Drive will eventually be coming to SharePoint on-prem as well. They use the SharePoint framework, so if they’re building their stuff on top of the SharePoint framework, that’s considered what we call “first party.” We third-party developers and people that are out there and who are going to be, and when I want to build something for SharePoint as Andrew Connell, I’m third party. So, I’m using the same framework as Microsoft is using to build their own stuff. I know that they are going to be adding more and more features to it. As they add more and more features to it, I get to take advantage of those as well. This is just the direction that they are going. So, should you convert the old server-side code to use the new SharePoint framework? If it’s stuff that you eventually want to run in Office 365 in future versions of SharePoint if you upgrade. Yes, it’s in your best interest. If it’s stuff that you don’t ever plan to go to Office 365, if you don’t ever plan to upgrade from 2013 or 2016, then you just got to make that decision. Does it make sense?
Chris Ang: OK, great. So, mainly you’re saying that if there’s a reason where you are likely going to the cloud in the future, then you would recommend to look at the SharePoint framework for its advantages. Otherwise, it may not be completely necessary if you’re going to stay on-premises?
Andrew Connell: Yes, exactly. You look at it from greenfield or brownfield development. “Greenfield” meaning a “brand new thing.” I would first go to the SharePoint framework, because that’s what Microsoft is investing for—SharePoint Online, SharePoint 2016. For “brownfield” stuff, which means that I already have an existing environment and I’ve got some existing solutions. Should I migrate it? It’s like every migration. Let’s look at everything we have. Does it make sense to migrate? Just leave it where it is. Do we want to take advantage of just redoing it where it is now and just adding new features and things like that. Then, maybe we do want to go through and use this new framework. So, it’s all a case-by-case thing and the classic SharePoint answer—“it depends.” That’s the direction I would go.
Chris Ang: Yes, that makes sense. And again, for anyone that’s just joined in there’s a question section, so if you want to post some questions directly for any of the experts, you can post the questions and we will be getting to that shortly where we’ll try to answer them. OK, let’s get to infrastructure/administration/however you want to call it, but the infrastructure side we’ve got Vlad.
(Time: 23:27)
Infrastructure Cleanup (Vlad Catrinescu)
Question 1: For regular maintenance of SharePoint, what are your Top 3 Best Practices? What are the top advantages?
Chris Ang: So, the first question for Vlad is that I’ve got a SharePoint environment and I need to maintain it. What are the top three things would you say that I absolutely have to do to make sure that my environment is healthy or is making sure that it is not falling apart, like there’s patches. What are the top three things you would say that you have to do?
Vlad Catrinescu: Sure, so the No. 1 is to keep the devs like Andrew Connell in check because devs always break stuff. Now, I’m joking <>. I’ve been in Orlando and I want him to bring me to the SpaceX launches and everything again, so I have to be nice.
One of the things that I talk a lot with customers is, “What are your priorities?” Keeping SharePoint healthy and keeping SharePoint fast and keeping SharePoint secure are the top three things that SharePoint admins should take care of. One of the things that has gotten in a bigger and bigger importance lately is to make sure that your SharePoint stays up to date. We’ve seen this even this year quite a few times where there have been hackers that have hacked systems only because they are out of date. If you are on the latest patch and using the latest version of Windows server or Windows client, they would have never had that bug and hackers were able to get into like banks, government agencies, only because they are using outdated software. So, make sure that you are actually installing the latest updates for SharePoint. I know that SharePoint has a bad reputation of its reliability of accumulative updates. I mean it is absolutely true for SharePoint 2010 and 2013 Microsoft didn’t deliver the biggest quality cumulative updates. They’ve been fixing one bug and introducing three new ones, but with SharePoint 2016, I think it’s been more than a year so far, and there really hasn’t been any bugs introduced with public updates. The quality process is a lot better. They’ve been shipping stuff and testing stuff a lot more, so make sure that both on the Windows server side and other SharePoint site that you’re keeping up to date with the latest security patches so you don’t get hacked. You’re probably hosting a lot of sensitive data on SharePoint and you don’t want to be the next company in front of the newspaper with documents that have been leaked to WikiLeaks. That’s number one.
Number two is to make sure that you look at those reports. SharePoint has a lot of useful reports built in now, so a lot of third-party tools are able to tell you how your infrastructure is doing and how everything is doing. I know a lot of people who never go to their search service application to look at their search crawl logs. Sometimes when I go out as a consultant, I go to make a health check. Then I realize that out of a million documents you have in SharePoint, you have 300,000 errors, and those are actually not only that people cannot find them, and as you know, searches how people find things those days, so basically nobody looks at those documents anymore and they probably upload them two times or three times because they think it’s not there. It also makes your crawl database and a bunch of databases bet huge because it’s storing all of the 300,000 errors every day when you do a crawl or every 15 minutes because it’s not able to find them. So, make sure that you look almost every day. Do I have any new errors in the event viewer? Do I have any errors on my network? Do I have any errors on the SQL Server? Do I have any errors anywhere, like in the search service application? Make sure that you monitor those. Make sure that you look at them and understand and you fix them as soon as possible. There’s a lot of SharePoint errors, especially they are out-of-the-box errors. Whenever you install SharePoint and you don’t even have a site on there, you already have 25 errors in the event viewer in the health check. So, I know it’s not all of them are mandatory that you look at, but make sure that you monitor because there is some really important stuff there and sometimes you’ll see that. A site collection went from one gig to one terabyte overnight. This can have a huge impact if you don’t monitor it. This can have a huge impact when your SQL Server drive or your host databases get so full, it might all go down. So, make sure you monitor your SharePoint.
Lastly, this one is more for regular maintenance as well as the security, and I know that Liam will talk a lot more about security, but make sure that all of your stuff is secure. Make sure that you HTTPS all over the place. A lot of companies that I still go to, they still use HTTP and they don’t really care about security because that’s inside the network anyway. Make sure that you secure everything and make sure that you document everything as well. You never know when you have a new person coming in and your SharePoint infrastructure is not documented or you don’t play with SharePoint every day and you’re not a dedicated admin. Make sure that you document all of the changes. Make sure you document when certificates expire or when other stuff in your infrastructure expires. Make sure you have all that documented as well as procedures of what happens if how do we change it. What if we have a solution deployed, where and what is affected, so if I ever do an update I know exactly where I have to update it.
Those are kind of my top three things.
Chris Ang: Great. Thanks, Vlad. So, first is mainly patching and making sure that your environment is up to date with the patches. I know you said a lot more, but I’m just trying to summarize. This webinar is going to be recorded, so anyone can play it back to hear all of the answers. No. 2 is to make sure to read the event viewer and logs. There are many environments where no one reads the logs, and there’s plenty of errors, and people don’t know what the users are going through if they can’t find things or things, so that’s a good one. No. 3 is make sure that you use HTTPS and make sure that you’re documenting your environment. That’s another thing that a lot of people overlook, I think.
Is that right, Vlad? I got those right?
Vlad Catrinescu: Right.
(Time: 30:32)
Infrastructure Cleanup (Vlad Catrinescu)
Question 2: What are some of the best practices around managing Content Databases?
Chris Ang: The next one that we have for you, Vlad, is around content databases. What would you say are best practices in maintaining and cleaning up content databases, large ones? What do you look at when you look at content databases?
Vlad Catrinescu: I actually look at a few things. First of all, I know that SharePoint 2016 supports databases up to 1 terabyte with no problem, It’s still within the offer, but the best practices, however, is to try to keep them small. When you keep your database really, really big, it will make it a bit slower and taking a back-up of your database. I’m sure that most of you take a back-up on the SQL side and not on the SharePoint side. If you take a database of a 200 gig database, it’s much faster to take the back-up. It will be easier and if you ever want to restore it, it will restore a lot faster. If you ever want to move it to a queue environment for testing after or for a refresh, it will be a lot easier to do, so even if Microsoft supports huge databases, the actual remit for inactive site collections is 4 terabytes. Try to keep the databases under 200 gigs. I know that this was the recommendation up to SharePoint 2013, but I still try to keep my database low. If I ever have a database two times at 500 gigs, and I see that there’s 20 site collections in that database, you can actually move site collections to a different content database with PowerShell. You can use the MOVE-SPSITE and then you can create new content database. Of course, creating new content databases is free. It’s not like you have to pay. It’s a lot simpler to keep large site collections in their own content database. Make sure that it’s a lot easier to back up or restore them or if you ever have to do any maintenance on just that content database, it’s a lot easier to only be specific to a site collection. If you migrate without a migration tool, whenever you migrate the SharePoint way or the Microsoft way, you migrate one content database at a time. So, by keeping them small, you can actually migrate one site collection at a time. If you want migrate two sites at a time or not everything, it will be a lot easier to select the sites that you migrate when it’s time to migrate everything. That’s No. 1.
No. 2 is to make sure that you have some SQL maintenance script to do your back-ups and whenever you do a full back-up, you string the log, or you index the database. You have all of that stuff in there, because if you don’t take care of your databases and you don’t back them up, a lot of the content databases are set up in full recovery mode, so the actual database logs will fill up and it will take a lot of space. After one year, you’ll look at it and you’ll say, “I have one terabyte of database logs. How did that happen?” So, make your monitor your databases. Make sure you have your SQL maintenance plan in place, so all the data is where it should be. Make sure that if it were needed. For example, if you have a lot of boutique groups, they all need to be on full recovery mode. If you’re not doing, for example, “I have a lot of clients that SharePoint is not business critical. That’s fine. You know, I love SharePoint, but it’s fine because it’s not always business critical. They only take SharePoint back-ups once a day because that’s what they need for their system. If you don’t take log back-ups and only take database back-ups once a day, you don’t need to have full recovery mode. To put it to simple recovery mode and if you’re not doing database back-ups, you’re not going to need that log file anyway. Also, pre-grow your databases, and that’s one of the things I say for optimization is that make your model database and make sure when you create the new content database, you can actually pre-grow it. By default, when you create the new database in SQL, it will take the size of your model database, so if your model database size is 8 mgs, which is by default, your new content database will be 8 mgs. If you don’t want to change model database, simply go into the content database and make it maybe 1 2 3 4 5 gigs from this start, depending on how much content you plan to have in the first six months. This way, the database does not have to auto-grow. SQL has this mechanism that wherever the database reaches its size, it will adjust the auto-grow mechanism to actually grow more to have space for new files. If you have a 100 mg database, your auto-grow is set at 1 mg and someone uploads a 10 mg document, the database will have to pre-grow 10 to auto-grow 10 times before that user is able to save that document into SharePoint, so that will make everything a lot slower. Make sure to go to your content databases and make sure to look at what their current size is and how much free space is in there. If you see that there is not a lot of free space, pre-grow them, so in this way they don’t have too little grow and it will be ready for people to start adding documents and collaborating without waiting on SQL Server to auto-grow databases. That’s about it.
Chris Ang: Thanks, Vlad. That’s a lot of good information. So, to quickly recap. I heard that to help maintain your content databases and to help clean them up, you recommend that although the limit has been increased to 1 terabyte or higher, to keep it under 200 gigabytes and use PowerShell to move SPsite or site collection to another content database. Number 2 is to make sure that you have your SQL maintenance plans running on schedule and that you’re doing back-ups to make sure that the logs get shrunk or truncated <<??>> (Time: 37:08). Number 3, you can use a model database or make sure that you set the row by database characteristics and settings. You can pre-grow databases for when you are creating new ones to help with maintenance. Does that sound about right?
Vlad Catrinescu: That sounds really perfect.
Chris Ang: Great! So, one more poll. What are your top clean-up priorities? When you are thinking of clean-up, what are the top clean-up priorities for you? We’ve got permissions and security. Is that something you’re really concerned about with clean-ups in your environment? You’re concerned with large content databases and don’t know how or need help managing that. Your back-ups are slow, etc. Pre-migration and server-side code, so when you’re migrating you may want to look at old server-side code or pre-migration is something you’re working on right now that you may need help with or some advice, clean-ups, or then we’ve got another option for performance. Cleaning up to optimize performance for the end users. So the votes are coming in and I’ll be sharing this in a moment.
Joel Oleson: It looks like we have half of the people voting. It looks like permissions and security is leading the pack, but it’s among the rest that are pretty neck and neck.
Chris Ang: OK, we are past 70 now. I’m going to share this. “Permissions & Security” is the biggest concerns. “Performance” kind of creeped up there, and it’s a bit more of a concern to clean up to improve performance. I want to make sure that we make use of the time that we have left to get to the questions from everyone.
Again, if you just joined, you can post some questions in the questions area.
(Time: 39:52)
UI/UX Cleanup (Joel Oleson)
Question 1: What are the best practices for keeping an intranet relevant from cleanup perspective?
Chris Ang: Now our next is for Joel. UI and user experience in clean-up. The first questions is what is your best practice or perspective for clean-up on keeping an intranet relaxant.
Joel Oleson: I think this is one of the areas that people forget about is the fact that the intranet and the home pages and on your various top-level pages, the people have to use that. If it’s not interesting and engaging, there are really some things you can do to optimize that. People complain about not getting adoption, and the first thing you notice is that they’ve got the same lists or the same library or the same static web part or the same images that don’t rotate or don’t change. So, my recommendation is to analyze it. Is this information useful? Is it relevant? We’ve talked about relevancy and getting adoption, but if you’ve got a bunch of static content, it doesn’t matter whether it’s the intranet or your SharePoint sites, there’s so many web parts that have been created that are simply not being used by teams to basically make your site more interesting and engaging. I think that the new communications site, as an example, is one that’s designed around being able to show the new news and show what is going on in a site. Imagine if top-level site has the same news from two years ago. As soon as somebody visits that site, they notice that it’s not even relevant and someone is not using this site. People are going to turn off this site and stop following this site because it is not relevant. The funny thing is that two levels down there is some list that is going nuts, and so really there’s a lot going on, but if your pages are not reflecting that, you’re going to have adoption issues and you’re going to have people that are going to just dismiss SharePoint and say, “Well, SharePoint doesn’t show me relevant things. It’s old and needs to be archived.” The idea of keeping things fresh is the other part of it. Building eye candy is kind of the first thing in making it relevant and dynamic and keeping it fresh. Sometimes people will take the responsibility off of themselves, but it is better to get the departments and divisions sharing that burden. Very soon, it does not have to be a burden. There was a guy once on our intranet and he said, “I hate our news site.” So I asked him what he meant that he hated it. He said that he had to create news once a week and it’s the worst part of my job. I’m thinking, if you hate it, what do you think the people reading your news are going to like. Stop, stop, stop creating news that is not relevant—that is, news that you hate and you’re just doing it to do it. People need to be writing about the stuff that they care about. Anyway, I’m sure that I could go on and on, but it’s a lot about fresh and dynamic and focusing on things that can be automated to keep showing the documents that are new. Show the fresh side of the SharePoint site for your intranet.
Chris Ang: OK, thanks Joel. Mainly, there’s a lot of high-visibility clean-up and that you’ve got fresh and relevant information there. Clean up the old archived documents. Get rid of the old content and make sure you have content that people can engage in. I hope I did not miss anything. I think that is the coolest thing.
Joel Oleson: If there’s web parts built into the product. As an example, content query, the search web parts. There’s a lot of stuff that is in the box that can help you get there. In addition to all of the third parties that can help you build more eye candy type of stuff.
Chris Ang: OK, then next we have is that when doing an upgrade, there are so many new apps that come out from Microsoft and certain third-party tools. Even with Microsoft apps, there’s flow and there’s Microsoft teams and planner and so forth. When using in SharePoint and there’s a new technology that we might want to move to to replace old ones, what would you suggest would be the approach?
(Time: 44:09)
UI/UX Cleanup (Joel Oleson)
Question 2: When doing an upgrade, what is your strategy on taking advantage of the new technology? And cleaning up old ones? For example, flow vs. classic workflows?
Joel Oleson: That’s a good question. Essentially, you’re going to the new version. The first thing people want to do is pick it all up and dump it into the new version. Often with customers, I like to say, let’s do an “art of the possible.” Let’s actually explore the new technology and talk about how you want to take advantage of it. Let’s actually build the destination and then we can decide what we want to pick up and move into it. If you simply take your 2013 and dump it into your 2016, guess what? It’s going to look and feel exactly the same, and it’s not even going to feel like a new version. It’s going to be a painful experience. You’re going to hate it. What is much better is that you build out your destination. Maybe you’re looking at one of those new fancy communication sites or maybe you’re looking at an intranet in a box from one of the dozens of third parties that builds a really nice new and exciting intranet built on top of Office 365 or SharePoint 2016. Then you’re saying, what do you want to put in there? When you have those web parts that bubble up technology, we’re really looking at good stuff. Let me give you one key thing. As an example, you’re going to keep pushing your old InfoPath workflows, and the poor experience they have in terms of it doesn’t work on this browser or it doesn’t work in mobile or I can’t print it to a .PDF, there’s so many things that are wrong within InfoPath. If people could just leave it in the grave or let it die and move out of it into flow and power apps and any of those third parties that are out there, you’re doing to have a much, much, much better experience than just saying we are just going to upgrade it and then get to it when we can. The forklift scenario to pick it up and dump it has got to go. Especially teams. How many teams are now using teams rather than using old SharePoint sites, and if they’re not, how can you map it to that strategy or leveraging teams? So much better. My team barely uses SharePoint in terms of collaboration. We use it as an intranet platform, but for the most part, our team does all of its communication in teams now and the documents automatically go into SharePoint. It’s a much better use of the technology.
Chris Ang: Great. Thanks, Joel. So, stay away from the big forklift and just take everything from your old and dumping it into your new just because it’s new and kind of take a step back and analyze this new technology and what you can take advantage of and where the technologies are definitely beneficial to replace things like InfoPath. When you’re moving, stay away from just dumping it into the new platform.
Joel Oleson: Exactly. I think you’re just shooting yourself in your own foot when you say that we’re going to just move it and then we’ll think about it. It’s a really bad strategy to say that we will get to it eventually. SharePoint is going to continue to be the thing that everybody points at and says that this is no good and just a place where people dump their stuff. Upgrade is that opportunity to get it right each time. Let’s start with what we want and move our stuff into that structure.
Chris Ang: Great. We are running a little bit short on time now, but these are some great answers. For security, Liam, what would your approach be for cleaning up permissions? I know this is a hot topic, but what would your approach be, and I know this is kind of generic, but where would you look first—item level for performance reasons or users with direct access? How would you do it and what’s your experience with this?
(Time: 48:15)
Security Cleanup (Liam Cleary)
Question 1: How would you approach cleaning up permissions?
Liam Cleary: Yes, those of us who have used SharePoint for a while know that the out-of-the-box tools are not the greatest. In reality, in order for me to get a list of who has permissions wherever, there’s nothing fundamentally in SharePoint that lets you do that. Obviously, you can use the user check, if you’re concerned about users, because realistically that’s where the permission issues occur. We follow the model that was given to us, which was to delegate site ownership to individuals and then we hand them the keys and say “knock yourself out and you look after and don’t bug me IT, because I don’t want to keep managing permissions.” Then, we reach a point where we get help desk calls and people can’t get in or they’ve lost access or someone has access where they shouldn’t. So realistically, you’re starting at every single layer in SharePoint, which is a pain because there’s not really one point that you need to focus on. It’s broken its inherited permissions, too, because they could be wrong. Realistically, the only way of actually doing it is that you either go through manually, which is a pain, or you use something like PowerShell. PowerShell is great for this. If you are looking for tasks that can be repeated, PowerShell is fantastic to iterate through and dump out a list of all of the permissions and then hand those permissions off to have them checked by the owners of the site. Or, you can use third-party tools, like the tool that you guys have got. That does the same thing, but that only covers the list of getting stuff out. The fixing right now is still a manual process. If you find a list or library that has broken inheritance that shouldn’t have and permissions have been added to individual users, the only way of fixing that is to manually make a change. Again, you could use PowerShell or other components to do that, but that’s really the world that we sit in. It’s not the most pleasant thing we have to do to go and clean up permissions, because by the very nature of having the word “share” in its name, the ideas is that you just allow people to just “do their thing.” So, for me that’s where you start. Getting a report out of SharePoint first is your starting point, whether it’s manual, PowerShell or a third-party component, that’s where you have to start at every single layer in SharePoint.
Chris Ang: OK, great. So, what I hear is that there’s really no out-of-the-box permissions tool to get an overall picture. You usually use PowerShell to iterate through and get a picture of where you stand first. Do you have any suggestions for what to look at first or really it just comes down to “every environment is different?” Item level, should I be concerned with a lot of item-level permissions or too many users or that just really depends on the requirements. Do you think that just mainly you want to get an overall picture and use PowerShell to get an overall report, right?
Liam Cleary: Yes, obviously, the big concerns have always been in every version of SharePoint this unique permission where you’ve added individual accounts. I mean that’s always been an issue way back in the early days of SharePoint. I’ve got 50,000 people in the organization and Joe decides to add thousands of those into one file as permissions. There are things in this SharePoint stack that will break when you put too many unique permissions into certain places. That’s always a good start. Let’s look at the unique permissions. Let’s look at where people have broken inheritance, and then let’s try to fix those. The fix for those is that you go back to the model that you are supposed to use—have users in the Active Directory groups and you add those groups where they’re supposed to be. That’s where I like to start. Use the report that comes out and work backwards, basically from the bottom upwards so the lowest level that permissions can be assigned and work backwards that way.
Chris Ang: Right. I apologize to everyone that we are a bit late on time, so the webinar will go a little bit past the hour I know that some of you may have to drop off for meetings, but let’s keep going. I guess if you could quickly answer this one, Liam. For an overall strategy for cleaning up SharePoint, what would be the goal? Do you have any idea overall, using AD groups, for example, would be one. I don’t want to put the answer in your mouth. What answer would I strive for? Not using individual users? What would you say?
(Time: 53:16)
Security Cleanup (Liam Cleary)
Question 2: What is the strategy to make our security schema in SharePoint “good” / following best practices?
Liam Cleary: Historically, Microsoft has always pushed Active Directory groups. That’s going to be your boundary. Don’t add individual users, but in reality, that does not work. There are times that you do need to have smaller subsets. I’ve walked into clients before where they have 10,000+ Active Directory groups just for SharePoint, and that’s purely based on them having to create an Active Directory group for every type of permission that they wanted to have. Now that is great, but that meant that managing Active Directory was awful. That meant for them to do anything, it also meant that IT had to manage everything. In reality, the best practice is to do what Microsoft has always said. Have the users and have AD groups, but have the AD groups in the site groups and assign permissions in the site groups; however, don’t be afraid to put individual users into a site group where needed. It doesn’t make any sense to have AD groups and site groups and individual groups together, but where possible, if you do need to break inheritance or you do need to have unique permissions, then be realistic about it. If you need to have unique permissions for 50 people, go and create an AD group for them. If you need permissions for five people, you don’t need an AD group. Just go and assign the permissions where you need to. It goes hand in hand with the first question. Make sure you know that that’s what you did, because nine times out of 10, when I do an audit of an environment, no one seems to know why something was done. Even though your best practice would be AD users, AD groups, site groups, which ultimately maps into the way SharePoint online works, as well, you still need to know that you have those changes, so you can say “you broke inheritance here and this is the reason why.” It comes back to that governance and management stuff as well. That’s really the best practice. If you follow what it is—AD users, AD groups, site groups—and then break where you need it and then move to SharePoint Online and Azure AD, that’s exactly the model that Azure AD and SharePoint Online uses, so you’ll be in the best place to go. Just don’t go crazy with all these users. Beware, because that doesn’t bode well in SharePoint Online either.
Chris Ang: And going back to your answer for number one, I think that helps once you have some kind of mechanism for your PowerShell and some sort of reporting tool in place and you can rerun that to get an idea of where things and maintain things and keep things clean.
Liam Cleary: Yes, just to add that’s one overarching thing is too often what tends to happen is that you have someone come in to help build the SharePoint environment. It’s all good to go. It’s super. It’s got fantastic content. All of the permissions are right. Then, the next time someone looks at it and has an audit, it’s a year later. There’s a lot of changes that are made in a year. Trying to retrospectively go back and fix permission things that were done six months ago and you don’t remember why. It’s really, really complicated. So, I run this report when I worked with a client recently. They’re out of Tampa. They run a set of PowerShell scripts every 30 days. Every 30 days it runs out a security report. It gets sent to the owners of those areas and then they make changes at those 30 days. Of course, understanding that not everyone has the resources to do that, but with something that can be automated, it’s a piece of cake. I would suggest that at least every quarter to have an audit done so that you can capture these issues. There is nothing harder than finding someone who has permissions that shouldn’t have permissions and then they take content that they shouldn’t have. My little thing, Chris, before I let you go is (and it sounds funny) but just because the CEO says he wants access, tell him “no,” because he may not need access. That will be your biggest win in controlling some of the security issues that I see. When someone says that they need full control to that, the answer is “No, you don’t. Explain to me why you need access.” I know it’s a hard conversation to have, but you’ll save yourself the hassle in the future.
POLL
Chris Ang: OK, we’ve got another poll here. This will be recorded, so some of you might need to drop off, but this poll is on the upcoming episodes, what would you like to address? What do you think is the most important to you? We’ve got what are the most powerful favorite features of SharePoint 2016, maintaining a hybrid environment—Office 365 and on-premises; Microsoft Flow and PowerApps; artificial intelligence and SharePoint and how can I leverage it. There’s a big buzz around that right now, so is there anything there; and then if you have any other topics you’d like us to discuss in the next episode, please post it in the questions.
OK, so that’s almost 70 percent. OK, great. So mostly, SharePoint 2016 and a close second is Microsoft Flow and PowerApps, and then we’ve got maintaining a hybrid environment.
QiPoint Product Mention
Now I’m going to try selling you on some of the products. I’ll promise to make this quick.
We have a toolkit called SharePoint Essentials Toolkit that will help you with your clean-up. Some of the top features are no server install, a built-in scheduler, and export reports automatically to SharePoint like these permission reports, item-level permission reports, list inventory and site inventory are many reports that you can use in the tool for helping you with your clean-ups. For example, there’s a spot to fix broken links in your environment or finding files that are old or large. If you need to find old web parts, you can build a report on where those are located. There is a spot for site and list clean-up, so finding old sites and getting a huge list of all of your sites without knowing PowerShell and you can see who has last modified or when the site was last modified. Finding old sites, large sites, and bulk deleting sites.
Joel Oleson: Hey, Chris, it looks like there is a question in the Q&A that relates right to your product even. It says, is there a way to find broken links to anything links the documents, images that I want to move or delete.
Chris Ang: Yes, there is a component in the SharePoint Essentials Toolkit called “Broken Link Manager.” After doing a migration or even just doing a clean-up, you can find the dead, broken links inside of pages and web parts, inside of files. Even goes inside. It has a very deep scanning capabilities, so it will find links inside of files and .PDFs, Word documents, metadata in your navigation, and it will also do an automatic find and replace, so you can put in the find and replace rules. We’ll have a webinar tomorrow as well as next week. You’ll get an email invite and if you want to participate in that. It’s called “Broken Link Manager.” It’s on our website as well. Thanks, Joel, for pointing that out.
Joel Oleson: I have another one as well. Let me just give you this as well. I think it will help you out as you’re covering your stuff. “I’ve customized a lot of pages in designer. Now I hear they won’t cleanly upgrade. How can I find all of them to get ready for upgrade?” I imagine this probably works for a lot of things like a web part on there that I want to remove from a lot of pages or I’m sure there are a lot of scenarios where you’d like help to find this thing, such as JavaScript snippet or whatever.
Chris Ang: Yes, so we have an item query tool that is a couple of slides back. That will allow you to find web parts and scripts even. You can paste the text. It supports regular expressions. It will build an Excel-type tabular report with all of the locations of all of these matches that it will find, so it will parse inside of the documents as well as even looking for an old CEO’s name or the departmental name changes, scripts, etc., so old error or you’ve migrated and have old web parts that show you errors and cannot be displayed, you can find all of those in a single report. We’ve got the permissions here, like Liam was explaining. It goes across sites. I’ve worked with SharePoint quite a while but not as long as Joel, of course. This is always an issue. I can write PowerShell, but not everyone can and update it. So, this tool will allow you to do that.
So, we have a short demo after, but I want to get to the questions, but some of them we have tried addressing while we were speaking. So, Joel, I’ll had this over to you, and we’ll try to answer some of these for five minutes or so.
TIME: 103:47: OPEN QUESTIONS
Joel Oleson: No problem. I’m going to pick and choose some random ones. Someone says that they got a new job as the SharePoint 2013 admin and they’re wondering if there’s any ideas on what they should do first. They’re saying that they do not use PowerShell and do not know what to do first. It has not been managed for two years. Sounds like a great clean-up question. Where do they start? Anyone want to jump in from the panel? Vlad?
Vlad Catrinescu: Of course, somebody had to remain on mute and talk without that. <> OK, when you’re a new SharePoint admin, what do you do? First of all, try to find out how people are using SharePoint and for how long they’ve been using it. How important is this stuff that they have under? After that you may find out they did not even know that they have SharePoint farms or five SharePoint farms that they need to take care of. Make sure that you talk to people to see what SharePoint site they use and try to document what the infrastructure looks like in the back. The other thing that I do is always look at the back-up settings, because I don’t want to be the new admin and then on the second day have someone come to me and say, “Hey, I’ve been using this site for a while and I really access it today since someone has deleted it or I deleted this by accident and I want to restore it.” So, one of the things that I make sure is that proper back-up procedures are in place, so if anything happens I can restore it and it’s not going to be a crisis. Other than that, try to see if there is any clean-up you can do. Maybe this is the chance to actually migrate to SharePoint 2016 and also clean it up, or if you’re staying on 2013, make sure you update to the latest updates or latest cumulative updates. So once, you are up to date.
Joel Oleson: Thanks! This next question is the best practices for site collection design hierarchy for SharePoint Online and I want to make these a little bit quick, so I’m going to give a quick answer. I’ve found that the site collection object is a great object. If you’re doing collaboration, use site collections. Don’t go super, super deep. If you’re doing intranets, keep it contained within a site collection unless you’re getting really large. Microsoft has to worry about your database since you’re online, but ultimately their much easier to move around if you’ve got them in site collections, even with SharePoint Online. In terms of hierarchy, like I mentioned previously, build out that hierarchy and then move into it. That is a good approach.
Here is one for Andrew. Tell me about cross-site publishing and when best to use them. Here is an example, to them cross-site publishing may be new. What’s a good example of when to use that?
Andrew Connell: I might actually be a bad person to answer this one. <> I haven’t done this or looked at this type of stuff in many, many years. I’d probably defer to you or Vlad on this one.
Vlad Catrinescu: So cross-site publishing, my opinion of the time when it is successful is when you want to have a public site and an internal site or like this site that is people-facing in it. I’ll say a QA site or a site people are using and then you want to have a lot of people enter content on the actual QA site or are not the production site and then you want to have different rules on when the content gets published and what status does it need to have that you need to go to like 10,000 modifications, and then when it gets to a certain status, it gets published on the actual site, so that’s one of the use cases when I would actually use it.
Joel Oleson: And I would say there’s Microsoft saying, “here’s the feature and here’s how it works,” and then there’s the reality. I really haven’t hardly seen anybody use this particular feature because replication and SharePoint really don’t exist. When it comes down to it, when it’s like “oh, that sounds fantastic. I make a site and have it go out to the extranet and keep those in sync.” That’s not how it works. That’s not how reality works either. If someone was truly looking for replication, they need to be looking at a third-party product. If you’re trying to say one time, then you’re talking about migration, and there’s better migration tools than cross-site publishing, so you’ve got to be careful. If it’s just a list, then maybe you can get away with it. In reality, I don’t see it. It’s kind of funny, but I don’t see it.
How do I remove deleted users from site collections but not disabled users? I was able to do that on ControlPoint, but now I’m not able to do that on SharePoint Online.
Vlad Catrinescu: A lot of fun PowerShell, I would say. Without getting too technical, you’re going to use the SharePoint Online module and Azure AD one. You’re going to have to go look for every site collection. Do a GET SPO USER for every user that’s there. Just do a check to see if they still exist in AD. If they don’t exist in AD, then delete them please from SharePoint. That’s about it, but PowerShell is your answer.
Joel Oleson: Yes, I was wondering if Chris had an answer as to how they might do that with a tool.
Chris Ang: At the item level, you can use the tool to parse files. That’s what they are trying to do.
Joel Oleson: Trying to find users like deleted users and not disabled users. I think it’s some kind of cleaning up the site collection from these people no longer exist in the company but they’re still showing up. Some people they want them to stop showing. I think in SharePointland we like to keep them around, so we can say, “That person really did create that document, but they’re going to stay in the user info table.
Chris Ang: The user information list.
Joel Oleson: Some user information will stick around, even though they are not in the company, so they want to remove them remove them. They are disabling them in AD but you don’t want to leave the permissions, especially if it’s an external-facing environment. Some people may be worried about it, even though it does not do anything.
Chris Ang: The tool has two reports that can do it. There’s one aspect where it’s the permissions report, so you can build a permissions report across the sites and gather the site-level, list-level, item-level permissions and then you can filter. You need to know who the users are for that report that you’re looking for that might not be there anymore. You can then filter that report by those users and it will show you where they are located. Another one is called the orphaned user report. That’s part of the SharePoint Essentials Toolkit that will allow you to find ones that are disabled or deleted from Active Directory but are present in SharePoint. There’s the item-level report. If it’s in the user information list or you have references to that user, the item-level report can search for a user by a display name or user name, you just enter the text that you’re trying to search and it will match metadata as well as the people picker field, if you need to find them like that.
Joel Oleson: One of the things I’m seeing in a few different questions I’ll try and kind of consolidate, is people who want to find content that is no longer being used—sites, site collections that really aren’t being used anymore they can now archived and recommendations there.
Liam Cleary: That’s going to be PowerShell. That’s the easy one if you’re going to use out-of-the-box or you could actually use SharePoint Essentials Toolkit, as well, that Chris is talking about that actually does the same thing. You have the ability to troll through the content the same as you would use if you were going to use PowerShell. You’re basically checking for dates of when things were done and then you’re initiating a delete or archive or whatever else. There’s two different ways of doing it. The SharePoint Essentials Toolkit does it really, really well. I quite like that the way it does it, because it means that I can keep running it over and over and over again, which I’m always a great advocate to just schedule it. Now, I’m not going through the effort of (like Vlad said) writing some majorly complex PowerShell and then never running it again. It makes no sense. Trying to have something that can be run again. Definitely the toolkit has that and it works really, really well. It finds sites, content libraries, lists that have been sitting around ages and then giving the ability to clean them up.
Vlad Catrinescu: Just o ad to what Liam said. You can do it with PowerShell, but it does not really work well. One of the problems with PowerShell is that it looks at a lot of the hidden stuff that you really don’t have access or care, so when you actually do get SPO site class content modified date, if Microsoft did not update on SharePoint Online, which happens almost every day or every (let’s say) week, I’ll be generous, the thing that it will update to last content modified date from PowerShell, so the information you get a lot of the times from PowerShell is actually not true, unless you do really complicated scripts. If you try to do the site class content modified date, that never works anymore. When I do that now, it always shows the same day, even if first sites I did not access in a year. So, SharePoint Essential Toolkit is definitely a lot better than using PowerShell.
Joel Oleson: Yes, I think PowerShell is so powerful that you make a change and there’s no rollback. You’ve got to be really careful when making changes. So, last, last one and we’ll have to save some of these questions for later and we’ll hand it back to Chris.
So, Andrew in some of his questions was saying to go SharePoint Framework if you can. This person is a little worried. They say how can we be sure that SharePoint is not creating security holes in our sites?
Andrew Connell: So, it’s a good question. I know that Liam and I both chimed in on it, but I would just rehash what I said further. (Time: 1:15:20) There’s nothing that SharePoint framework can be opening up holes you. This problem with this issue and however you want to look at it is no different in the SharePoint framework than it is with anything that’s custom development. When you’re building some of the custom code, you can easily make calls out to external sources or to have that consume stuff from external sources. The big thing is always going to be to do code reviews to what you roll out in production. There is no other way around it. There are technical solutions for this. You can put up certain rules and firewalls and block access—inbound access and outbound access. I’ve spent time working on a Defense Department server deck at the top-level security requirements that we have here in the United States. I’ve been told that it’s technically and almost impossible if you’re going to have any kind of external application or for your users to get out unless you put up a complete block. There’s no way to just technically prevent this from happen. To me, it’s going to be looking at where you are referencing stuff, making sure that with the SharePoint framework, we have things like CSS, JavaScript, that can be hosted in CDN so who has access to deploy and who has access to change that JavaScript. There is absolutely nothing stopping me from rolling out a new web part that I deployed to a SharePoint site that’s referencing a JavaScript file that’s living in an Azure CDN. It’s all deployed and without my customer ever know it, I could go in and change the JavaScript. Now, when the CEO goes to the site, I can then run my code under his access because he’s the one who is logged in so he’s the one who was actually running the JavaScript. I can go use the Microsoft graph or I could use the SharePoint REST API to go get data from anywhere in SharePoint. Now, the way I can stop that from happening is through a code review or making sure who has access to change that JavaScript file. That is more of a process thing that you have to implement. The technology really has nothing to do with it.
Joel Oleson: That’s good. Let me hand this back to Chris.
Chris Ang: OK. Thanks, Joel. So, thanks again, everyone. I apologize. We went over a bit. We had so much of that good information here. We have one more poll for those who are on here now. How should we follow up with you?
For the SharePoint Essentials Toolkit or for any questions that you have for any of the experts. What is the best way? What is the time frame that we could follow up with you or perhaps schedule a screen-sharing session or so forth?
Joel Oleson: I know some people who would like to see the demo or when they would like to see the demo. It does not mean that you have to be committed.
Chris Ang: Right. You’ll have an automatic email that you’ll get in about an hour that will have the links to the webinar, if you’re interested. Completely optional, obvsiously.
Joel Oleson: We will let that run for a while. We can do some questions while we wait.
So, someone says, scanning more than 20,000 for broken links replacing them, would that affect the SharePoint Server performance.
Chris Ang: Using the toolkit, the toolkit is using web services. It’s using CSOM, so when it’s reading files and opening files, it caches it into memory on the local machine under the account that you’re using. It’s similar to opening a web page like hundreds of times a second. So, it depends. If you’re SharePoint environment is already a little shaky and at its max, using the toolkit to scan for broken links for 20,000 files, typically, it won’t have any performance degradation. Do your testing and compare some more to opening files on your web browser because it’s going to open the web pages, etc., to read them for parse.
Joel Oleson: I would say that a lot of this comes down to, when you’re talking about performance, it does not matter if you’re talking about PowerShell or the tool using CSOM, it comes down to how many threads. If you’re writing your own code, you can manage the performance impact based on the number of threads, so that’s basically how you can manage the performance. If you’re doing one thread or 10 threads, you’re not going to notice it on the server. So, you know recursively, but you get the idea.
I just want to thank everyone on the panel who joined us today. Thanks for hosting this, Chris.
Chris Ang: Oh no problem. Thanks for joining. It’s great to have such an all-star cast here to give all of the different angles. Joel and I thought it would be a great time to get a great overall view of certain topics. Thanks.
Joel Oleson: Thanks to the audience. We got more than 50 percent who stuck with us for the extra half hour. If you want to end with that video, I think we probably could.
Chris Ang: Yes, we do have a video that we have to share the SharePoint Essentials Toolkit to show a few things. Liam was kind enough to record it and demonstrate some of the features of the tool, so let me start that. (It’s not really letting me start this right now. I think I have to stop the poll just a minute.) Here we go. This is going over some of the features.
DEMO VIDEO MENTIONED BUT NOT PLAYED.
- Thank you everyone for staying on. It’s still a lot of people still on so I hope you found it useful the demonstration that Liam did of the SharePoint Essentials Toolkit. For our future episodes, planning to schedule them every 2-3 months. You will get an email to join future episodes. The website is on the screen. We will also post some free PowerShell scripts to help you with clean-ups. It will build an inventory of your site into an Excel file format (.csv file) with all site collections at once, so it will be to run entire. It’s two scripts—one for SharePoint online and one for on-premises that we’ll be providing. There will also be a free infographic that we’ll be posting, so you can use that if you need to, regarding SharePoint cleanups and some statistics about that. OK, we’ll of course, be providing the webinar recording on that site. All of the questions that we weren’t able to get to we’ll be trying to address them in a future episode and posting these answers into the site.
Thanks again everyone for joining. It’s been great to have everyone. Again, please visit our website and check out our tool. Subscribe from the website that was just on to follow The Expert Series. If you follow The Expert Series you’ll be able to get notified of all the future episodes.
This concludes the first episode of The Expert Series. Thanks everyone. It’s been a pleasure to have everyone on. Thanks!
SharePoint Site Inventory PowerShell Scripts
Build an inventory report on SharePoint Sites in your Office 365 and SharePoint on-premises environments. This is to help with your cleanup by reporting on sites, size, list counts, last modified etc..
Instructions:
- Download and then rename above file extension to ps1.
- Copy the file to your computer
- Right Click the file, click Edit
- On lines 83-85, change to use your Office 365 Tenant Admin URL (such as https://mycompany-admin.sharepoint.com), username and password. Enclose them in double quotes. Ensure you are using an account with permission to access all site collections and sites.
- Save and click Run button (play button)
- A CSV and log text file should be created.
Instructions:
- Download and then rename above file extension to ps1.
- Copy the file to your SharePoint Web Server or App Server
- Right Click the file, click Edit
- On lines 85-87, change to use your web application URL, username and password. Enclose them in double quotes. Ensure you are using an account with permission to access all site collections and sites.
- Save and click Run button (play button)
- A CSV and log text file should be created.
Currently working at Cognillo (https://www.cognillo.com), he has helped architect and develop SharePoint Enterprise products for customers such as the U.S. Navy, U.S. Army, U.N. Security Council of Netherlands, Australian Government, U.S. Dept of Treasury, U.S. Dept of Justice, Canadian Dept of Defense, Scotiabank, JPMorgan CHASE Bank, Intel, Ford Motors, Microsoft, NASA, DARPA, SNC Lavalin, Penguin Books, and more.
He is a proud father of 2, and when he has any spare time, he loves to paint portraits of his kids.
- What is SharePoint? A Beginner’s Guide to MS SharePoint Software - October 8, 2018
- Beginner’s Guide: Windows PowerShell – How to Use Tutorial for Dummies - October 8, 2018
- Remote PowerShell to Manage SharePoint on-premises - August 14, 2018
Very nice, the information is great
Thanks